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Abstract 


Performance  Based  Logistics  is  the  most  preferred  product  support  strategy  in 
Department  of  Defense.  'Using  performance-based  acquisition  methods  to  the  maximum  extent 
practicable  when  acquiring  services  with  little  exclusion'  is  mandated.  Although  PBL  should  be 
used  whenever  feasible,  few  documents  point  out  how  to  measure  the  feasibility  of  PBL.  The  aim 
of  this  research  is  to  fill  this  gap  by  answering  the  question  "What  factors  determine  the 
appropriateness  of  the  selection  of  PBL  as  a  strategy  for  a  specific  acquisition?" 

This  research  determines  the  factors  affecting  the  success  in  selecting  PBL  as  an 
acquisition  method.  Each  factor  is  examined  in  detail  and  built  into  a  spreadsheet  tool  which 
helps  assess  the  appropriateness  of  PBL  as  an  acquisition  method.  The  purpose  of  this  tool  is  to 
aid  in  PBL-related  decision  making  processes  and  business  case  analyses.  Various  questions, 
asked  by  the  tool,  will  let  the  user  have  a  more  objective  assessment  in  a  relatively  short  time. 
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AN  ASSESSMENT  TOOL  OF  PERFORMANCE  BASED  LOGISTICS  APPROPRIATENESS 


I.  Introduction 

Background 

Performance  Based  Logistics  (PBL)  is  the  determined  product  support  strategy  in 
Department  of  Defense  (DoD)  which  capitalizes  on  integrated  logistics  chains  and  public/private 
partnerships.  Not  only  it  is  suggested  by  Defense  Acquisition  University  (DAU)  but  also  PBL 
Implementation  is  mandated  by  DoD  Directives.  Besides  Federal  Acquisition  Regulations  (FAR) 
emphasizes  that;  performance-based  acquisition  should  be  the  preferred  method  for  acquiring 
services  (Public  Law  106-398,  section  821).  Agencies  should  use  performance-based  acquisition 
methods  to  the  maximum  extent  practical  when  acquiring  services  with  little  exclusion  (FAR 
37.102). 

Like  North  America,  South  America  is  trying  to  implement  PBL  as  well  as  some 
European  countries.  As  Bozkurt  and  Guducu  (2005,  p.49)  state  “Budget  constraints  drive  all 
government  entities  to  look  for  new  and  efficient  ways  to  fulfill  their  missions.”  The  idea, 
underlying  this  worldwide  phenomenon,  is  thought  to  be  basically  reducing  costs.  The  F-35 
project  is  supported  by  a  20%  less  maintenance  cost  expectation,  comparing  with  substitute 
fighters  (Sols  and  Venna,  2007).  Chile,  Ecuador,  Paraguay,  and  Brazil  are  trying  to  adopt  PBL 
with  the  purchase  of  the  C-212  and  CN-235  aircraft  systems  through  European  Aeronautic 
Defense  and  Space  Company  Construcciones  Aeronauticas  S.A.  (EADS  CASA)  with  likewise 
expectations  (Cebeci,  2009).  United  Kingdom’s  Royal  Air  Force  (RAF)  is  expecting  to  reduce 
the  maintenance  costs  of  E-3D  Sentry  Airborne  Warning  and  Control  System  (AWACS)  aircraft 
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by  12  percent  over  previous  similar  systems  (Sols,  Nowick,  Verma,  2007).  Not  only  cost 
reduction  but  also  performance  improvements  are  expected. 

Turkey  is  also  affected  by  this  phenomenon.  In  order  to  perform  effectively,  the  Turkish 
Navy  constantly  pursues  useful  and  efficient  practices  of  PBL  (Bozkurt  and  Guducu,  2005). 
Like  the  Turkish  Navy,  Turkish  Air  Force  (TurAF)  and  Turkish  Army  supported  PBL 
researches.  There  are  some  differences  between  U.S.A.  and  Turkey  legacy  systems  regarding 
acquisition,  but  PBL  becomes  more  popular  day  by  day  in  Turkish  Armed  Forces.  On  the  other 
hand  as  Bozkurt  and  Guducu  (2005)  state,  these  differences  are  not  seemed  to  be  a  deal  breaker 
to  implement  and  apply  PBL  in  service  acquisitions.  Turkish  Air  Force  published  a 
supplementary  document  to  increase  awareness  about  PBL  in  2010.  This  supplementary 
document  states  the  trend  for  PBL,  beginning  with  F-35  Joint  Strike  Fighter  project. 
(Hv.K.K.ligi,  2010)  As  seen  PBL  is  still  a  newborn  baby  for  TurAF  and  there  is  a  long  way  to 
go- 

There  are  a  couple  of  studies  that  show  the  popularity  and  application  of  PBL  in  Turkey. 
Bozkurt  and  Guducu  (2005)  presented  the  differences  in  legacy  and  recommendations  about  how 
to  implement  PBL  in  Turkey.  Cebeci  (2009)  examines  the  success  of  perfonnance-based 
acquisition  in  non-westem  countries  by  exhibiting  the  cultural  aspect,  differences  between  non¬ 
western  countries  and  the  ones  applying  PBL.  Denizer  (2007)  provides  recommendations  to 
improve  Turkish  Anny’s  weapon  system  support  and  maintainability  based  on  findings,  and 
introduces  potentials  through  the  implementation  of  PBL  practices  for  the  Turkish  Army.  Turkey 
wants  to  develop  its  knowledge  about  PBL. 
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Problem  Statement 


As  stated  before  ‘application  of  PBL  whenever  and  wherever  it  is  possible’  is  mandated. 
In  opposition  of  these  statements  and  regulations,  there  is  not  much  regarding  the  feasibility  of 
PBL  or  the  factors  affecting  to  select  the  best  acquisition  strategy.  In  his  memorandum  for 
Secretaries  of  the  Military  Departments  Directors,  Defense  Agencies  Director,  Defense  Logistics 
Agency  J.  S.  Gansler  (The  under  Secretary  Of  Defense  -  Acquisition  and  Technology)  stated  that 
“It  is  the  policy  of  the  Department  of  Defense  (DoD)  that,  in  order  to  maximize  performance, 
innovation,  and  competition,  often  at  lower  cost,  performance-based  strategies  for  the  acquisition 
of  services  are  to  be  used  wherever  possible.”  But  also  he  admits  that  not  all  acquisitions  for 
services  can  be  conducted  in  a  performance-based  manner,  which  should  become  the  exceptions 
although  the  vast  majority  can  (The  Under  Secretary  Of  Defense  Memorandum,  2000).  But 
which  one  should  be  the  exception,  as  it  is  mentioned  in  this  Memorandum? 

In  their  study;  Cohen,  Netessine  and  Kim  (2006)  admits  that  it  would  be  wonderful 
paying  for  our  car  only  when  it  works,  or  our  television,  or  even  our  high-end  toaster.  They  have 
been  convinced  that  perfonnance  based  systems  are  becoming  widely  accepted  as  an  important 
component  of  a  new  strategy  for  the  management  of  after-sales  service  supply  chains.  They 
decided  to  properly  define  PBL  and  to  determine  why  and  when  PBL  or  power  by  the  hour 
works.  Their  new  study  is  based  on  examining  the  relationships  between  suppliers  and 
customers,  main  drivers  of  perfonnance  improvement,  incentive  effects  and  the  requirements  for 
a  successful  PBL  implementation.  Regarding  this  point  of  view,  this  research  aims  to  fill  this  gap 
in  the  literature;  to  determine  which  acquisitions  should  be  fit  by  PBL  and  which  should  be 
exceptions  as  DoD  defined.  After  finding  the  factors  out,  the  researcher  tried  to  build  a  tool  that 
assess  the  applicability  of  PBL  at  any  specific  acquisition. 
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Research  Questions 

The  purpose  of  this  study  is  finding  out  the  factors  that  affects  choosing  PBL  as  an 
acquisition  strategy  and  create  a  conformity  assessment  tool  which  states  whether  PBL  is  a  good 
choice  for  that  acquisition  or  not. 

In  order  to  create  this  tool  the  following  research  question  was  investigated: 

•  What  factors  determine  the  appropriateness  of  the  selection  of  PBL  as  a  strategy  for  a 
specific  acquisition? 

In  addition  to  the  research  question,  several  investigative  questions  were  established  to 
support  the  study,  which  will  be  discussed  in  further  chapters: 

•  What  is  the  current  policy  to  use  PBL?  Regarding  this  policy  does  every  acquisition  fit  in 
PBL? 

•  Has  DOD  ever  experienced  any  failure/pitfall  due  to  selecting  PBL  as  a  strategy? 

•  How  do  these  criteria  effect  acquisition  regarding  PBL? 

•  Is  this  assessment  tool  usable  in  every  service-acquisition  decision? 

Scope 

The  assessment  tool  will  be  created  for  decision  makers  to  reduce  their  decision  time  in 
making  more  precise  decisions.  Therefore  the  ideas,  which  are  mentioned  in  this  study,  are  only 
from  the  decision  makers’  point  of  view. 

In  addition  to  that,  the  Turkish  regulation  system  does  not  allow  any  awards  option  years 
or  anything  beyond  fixed  price.  On  the  other  hand,  according  to  U.S.  regulations  the  agencies 
must  use  the  following  order  of  precedence  when  acquiring  services:  (1)  A  firm-fixed  price 
performance-based  contract  or  task  order.  (2)  A  performance-based  contract  or  task  order  that  is 
not  firm-fixed  price.  (3)  A  contract  or  task  order  that  is  not  performance-based  (FAR  Subpart 
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37.102).  In  addition  to  regulations  as  Gardner’s  (2008)  research  findings  stated,  PBL 
implemented  should  strive  to  achieve  a  fixed  price  contract  for  their  programs.  As  seen.  Fixed 
Price  contracting  is  suggested  by  law  and  it  is  beneficial  as  long  as  the  outcome  with  the  price  is 
well-defined.  This  study  does  not  include  incentives’  effect  such  as  award  option  years. 

Methodology 

This  research  is  based  on  a  literature  survey  conducted  on  the  papers  published  in 
connection  with  this  topic  since  the  advent  of  the  PBL.  According  to  the  literature,  factors  which 
affect  PBL’s  success  are  detennined.  A  scale  for  each  factor  is  created  for  the  tool.  In  addition  to 
that  scale,  weights  are  assigned  to  each.  User  may  also  give  custom  score  and/or  weight  to  each 
factor. 

There  are  2  types  of  scores  and  3  types  of  weights  used  in  the  tool.  Tool  evaluated  score, 
which  is  calculated  according  to  the  answers  given  to  corresponding  questions,  and  user 
determined  score.  Occurrences’  frequency  weight  comes  up  from  the  sample  literature;  author 
determined  weight  is  the  weight  into  which  importance  of  that  factor  is  taken  account,  in  addition 
to  occurrences’  frequency,  and  user  detennined  weight  are  the  customized  weight  by  user. 
Occurrences  in  the  sample  literature  are  presented  in  Appendix  A. 

The  tool  is  created  in  Excel  by  logical  functions,  and  algebraic  calculations.  The  general 
score  is  the  sum  of  all  factor’s  score.  According  to  this  score,  tool  gives  advices  to  use  PBL  as  an 
acquisition  strategy  or  not  and  also  points  out  to  which  factors  user  should  pay  more  attention. 

Five  different  examples  are  presented  at  application  section.  Data  is  collected  from 
subject  matter  of  experts’  and/or  program  managers’  studies,  opinions,  and  ideas  regarding  that 
specific  acquisition.  Due  to  confidentiality  reasons,  their  names  and  units  are  not  given  and  some 
of  the  data  are  disguised  within  the  range  of  evaluation  score. 
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Assumptions 

The  PBL  approach  must  ensure  compliance  with  all  statutory  and  regulatory 
requirements,  and  in  particular,  the  statutory  limitations  of  the  Turkish  legislative  and  statutory 
system.  Therefore,  it  is  assumed  that  such  a  willing  country  has  already  made  the  required 
arrangements  at  the  point  of  selection  an  acquisition  strategy  from  alternatives  including  PBL.  In 
application  sections  for  some  PBL  programs  it  is  assumed  to  have  an  exception  for  these 
statutory  limitations  approved  by  the  Turkish  Grand  National  Assembly.  But  these  exceptions 
are  assumed  to  be  present  at  a  minimum  level. 

Implications  and  Summary 

Constraints;  such  as  shrinking  budget,  decreasing  funds,  incentives  to  increase 
perfonnance,  etc.  force  all  countries  to  find  new  ways  of  acquisition.  Therefore  the  growing 
popularity  and  usage  of  PBL  attracts  all.  Turkey  is  one  of  the  followers,  too.  This  study  will  help 
Turkey  to  increase  its  knowledge  about  PBL  and  provide  insight  regarding  the  factors  affecting 
in  selection  of  PBL  as  an  acquisition  strategy. 

As  stated  before,  there  is  not  much  study  regarding  the  feasibility  of  PBL  as  an 
acquisition  strategy  although  to  implement  PBL  is  mandated  and  encouraged.  Therefore,  this 
research  is  significant,  because  it  is  one  of  the  few  studies  which  assess  the  applicability  of  PBL 
for  a  specific  acquisition. 

In  addition,  this  research  suggests  using  the  tool  in  decision  making  of  whether  to  use 
PBL  as  a  strategy  or  not  for  a  particular  acquisition.  Managers  do  not  have  much  time  to  make 
everything  in  order.  As  Kaplan  and  Cooper  (1997,  p.144)  stated;  beyond  all,  “the  scarcest 
resource  in  an  organization  is  a  manager’s  time.”  The  tool,  created  by  this  study,  will  save 
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managers’  time  in  decision  making  progress;  help  them  not  to  get  lost  in  lots  of  criteria  or 
making  wrong  decisions. 

The  introduction  chapter  provides  a  summary  about  perfonnance-based  acquisition  and 
states  the  research  problem  addressed  by  the  study.  Afterwards  methodology  used  in  the  research 
was  discussed,  simply.  The  next  chapter  presents  a  detailed  literature  review  regarding  PBL  and 
factors  affecting  in  the  selection  of  PBL  as  an  acquisition  strategy.  Chapter  3  provides  a  detailed 
description  of  the  methodology  used  for  detennining  factors/weights,  and  building  the 
assessment  tool.  Chapter  4  presents  application  of  the  tool,  data  collection  and  analysis.  Chapter 
5  includes  conclusion  of  the  study,  limitations  and  recommendations  for  future  research. 
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II.  Literature  Review 


Overview 

Eighty  one  documents,  including  articles  in  journals,  DoD  guides,  thesis,  surveys,  white 
papers,  etc.  are  examined  for  determining  the  criteria  in  selection  of  PBL  as  an  acquisition 
method.  These  are  explained  in  detail  according  to  their  frequency  in  this  section.  While 
reviewing  the  literature;  emphasis  is  kept  on  the  factors  which  effects  PBL’s  success.  The  author 
came  up  with  thirteen  factors;  which  are  obligatory  for  PBL.  These  factors  are  below: 

1 .  Reduction  in  total  cost  of  ownership,  life  cycle  cost 

2.  Presence,  determination  of  metrics,  performance  outcomes 

3.  Improvement  in  performance 

4.  Partnership,  strategic  alliance 

5.  Length  of  contract 

6.  Flexibility 

7.  Better  service,  best  value  creation,  best  practices 

8.  Successful  implementation  of  PBL,  PBL  experience  and  training 

9.  Innovation 

10.  Data  availability 

1 1 .  Candidates  and  competition 

12.  Properties  of  system 

13.  Up-front,  early  planning 

Many  authors  mentioned  risk  and  the  stage  of  PBL.  But  these  are  not  seemed  to  be 
factors  that  may  affect  the  success  of  PBL.  In  addition  to  that  their  effect  is  enclosed  by  the  other 
factors’  effects.  Risk  and  stage  of  PBL  will  be  discussed  in  details  at  the  end  of  this  section. 
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1.  Reduction  in  Total  Cost  of  Ownership,  Life  Cycle  Cost 

In  sixty  out  of  eighty  one  documents,  reduction  in  total  cost  of  ownership  (TOC)  is 
mentioned  regarding  PBL  and  mostly  it  is  stated  as  PBL’s  purpose.  DoD  mandates  the  use  of 
performance  based  strategies  for  acquiring  and  sustaining  products  and  services  whenever 
feasible  “to  maximize  competition,  innovation,  and  interoperability,  and  to  enable  greater 
flexibility  in  capitalizing  on  commercial  technologies  to  reduce  costs.”  (DoDD  5000.1,  2003, 
p.7)  As  seen,  one  of  the  purposes  of  using  PBL  is  to  reduce  costs  for  DoD.  DoD  PBL  Guide 
(2005)  cited  this  and  states  that  PBL  application  should  be  cost  effective  as  validated  by  a 
Business  Case  Analysis.  By  DoD’s  words;  “The  primary  objective  of  Defense  acquisition  is  to 
acquire  quality  products  that  satisfy  user  needs  with  measurable  improvements  to  mission 
capability  and  operational  support,  in  a  timely  manner,  and  at  a  fair  and  reasonable  price.” 
(DoDD  5000.1,  2003,  p.3)  DoD  wants  program  managers  to  develop  and  implement 
performance-based  logistics  strategies  that  optimize  total  system  availability  while  minimizing 
cost  and  logistics  footprint  (DoDD  5000.1,  2003,  p.7).  Therefore  it  can  be  stated  that  PBL  might 
have  been  discovered  under  the  budget  pressure  and  requirement  for  reducing  costs. 

Nine  of  the  eighty  one  documents  cited  DoD’s  statements  above.  Buyukgural  (2009, 
p.88)  also  adds  “The  main  promise  of  the  PBL  is:  enabling  and  sustaining  a  desired  level  of 
performance  with  a  reasonable  cost  via  the  innovative  approaches  and  the  cumulated  experiences 
of  the  contractors.”  Bozkurt  and  Guducu  (2005)  count  “savings’  achievement”  as  one  of  the 
main  purposes  of  PBL.  In  their  report,  Pettingill,  Knipper,  Gaudette  and  Gayer  (2004)  compares 
the  definitions  used  for  PBL  and  emphasize  on  Navy’s  usage  of  “reducing  TOC”  and  DoD’s 
“minimizing  cost  and  logistics  footprint”.  They  suggest  using  both;  “reduced  logistics  footprint”, 
and  “reduced  total  ownership  costs”  in  definitions.  For  Kratz  (2001,  p.  12)  it  is  a  key  element  of 
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DoD’s  performance  based  environment;  “program  managers  to  guide  continuous  improvement 
of  weapon  system  supportability  and  reduction  in  operating  costs  by  dedicated  investments.” 

Twenty  one  documents  supported  cost  reduction  as  a  factor  with  examples.  Table  1 
shows  the  examples  used  by  corresponding  documents. 


Table  1.  Examples  used  in  documents-  Reduction  in  Total  Cost. 


ITAS 

F-117 

J  STARS 

APU 

GE  404 

A/C 
Ti  res 

C- 17 

GSP 

Shadow 

F-22 

U.K. 

Tornado 

T-45 

F/A-18E/F 

FIRST 

KC-130J 

H-60 

FLIR 

HIMARS 

M-l 

Abrams 

JAVELIN 

U.K. 

E-3D 

Beilis  (2001) 

X 

Berkowitz  et  al. 
(2003) 

X 

Boehk  (2003) 

X 

X 

Canaday  (2006) 

X 

X 

Cebeci  (2009) 

X 

X 

X 

Gansler, 
Lucyshyn, 
Harrington  and 
Corl  (2011) 

X 

GAO  Report: 

PBL  (2008) 

X 

X 

Geary,  Koster, 
Randall  and 
Haynie  (2010) 

X 

Goure  (2009) 

X 

X 

Goure  (2010) 

X 

X 

X 

X 

X 

X 

Goure  (2011b) 

X 

Kratz  and 

Buckingham 

(2010b) 

X 

Mahadevia, 
Engel  and 
Fowler  (2006) 

X 

O wings  (2010) 

X 

PBL  Guide 
(2005) 

X 

X 

X 

X 

Sols  and  Verma 
(2007) 

X 

Spring  (2010) 

X 

Vitasek  and 
Geary  (2008) 

X 

Vitasek, 

Cothran  and 
Rutner  (2006) 

X 

X 

X 

Vitasek, 

Cothran  and 
Rutner  (2007) 

X 

X 

Vitasek,  Geary, 
Cothran  and 
Rutner  (2006) 

X 

X 

X 

Total 

4 

4 

4 

4 

4 

3 

3 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

TOW  Improved  Target  Acquisition  System  (ITAS) 

This  is  a  system  designed  for  light  infantry  forces.  It  is  improved  for  target  acquisition 
and  fire  control. 

“The  ITAS  increases  target  detection,  acquisition,  recognition,  and  engagement  ranges, 
using  a  second-generation  Forward-Looking  Infrared  Radar  (FLIR),  an  eye-safe  laser 
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rangefinder,  and  other  digital  components.  TOW  ITAS  provides  a  highly  mobile,  adverse 
weather,  day  or  night  capability  needed  by  early  entry  forces  to  destroy  advanced  threat 
armor  at  greater  stand-off  ranges.  ITAS  fires  all  versions  of  the  TOW  missile  from  both 
the  M41  ground  launcher  and  the  Ml  121  HMMWV  platform  and  provides  a  growth  path 
for  future  missiles.”  (Perfonnance  Based  Logistics:  A  Program  Manager’s  Product 
Support  Guide,  2005,  p.5-3) 

A  PBL  contract  was  signed  with  Raytheon  in  2001  for  logistics  support.  As  stated,  in  the 
PBL  guide  (2005);  TOW  ITAS  is  one  of  the  Reduction  in  Total  Cost  (RTOC)  pilot  programs  that 
benefited  from  the  PBL  efforts.  Table  2  shows  the  cost  avoidance  results  per  each  fiscal  year: 


Table  2.  Cost  Avoidance  of  ITAS  PBL  Program  (Berkowitz  et  ah,  2003,  p.92). 


FY2001 

FY2002 

FY2003 

FY2004 

FY2005 

Total  FY 
99-  TC 

CLS  (TY$) 

7.7 

10.8 

13.5 

12.9 

16.0 

601.1 

Organic  (TY  $) 

18.2 

27.4 

18.2 

21.9 

29.0 

944.6 

Cost  Avoidance 

10.5 

16.6 

4.7 

8.9 

13.1 

343.5 

CLS  Cost  Avoidance  Major  Contributors:  No  TDP,  Replenishment  Spares,  Initial  Spares 
CLS  stands  for 


contracted  period.  As  seen 
above,  there  is  significant  cost 
avoidance.  Defence 
Acquisition  University  (DAU) 


points  out  the  cost  decrease  per 
system  shown  in  Figure  1 . 


Fixed  Price  Portion  of  Contract 


$$$ 


CO 

l 


$ 


Contract  Price  I  1  Price  per  System  per  Year 1 


figure  1 .  Cost  per  System  for  ITAS  Program 
PBL  Guide.  2005.  n.3-23). 


According  to  Army’s  records  Goure  (2010)  states  that  ITAS  for  the  TOW  missile  has 
provided  $350  million  in  cost  avoidance  with  better  than  90  percent  availability  over  the  past 
seven  years.  This  means  approximately  33%  less  cost  than  organic  structure. 

F-ll  7  Nighthawk 

This  is  the  first  low-observable  stealth  operational  aircraft. 
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“This  precision-strike  aircraft  penetrates  high-threat  airspace  and  can  employ  a  variety  of 
PGM  against  critical  targets.  The  F-l  17  is  equipped  with  sophisticated  navigation  and 
attack  systems  integrated  into  a  digital  avionics  suite  that  increases  mission  effectiveness 
and  reduces  pilot  workload.”  (Perfonnance  Based  Logistics:  A  Program  Manager’s 
Product  Support  Guide,  2005,  p.5-2) 

Lockheed  Martin  has  Total  System  Performance  Responsibility  (TSPR)  for  the  F-l  17 
weapon  system,  implemented  in  1998  and  at  first  for  5  years.  This  is  also  one  of  the  Reduction  in 
Total  Cost  (RTOC)  pilot  programs  that  benefited  from  the  PBL  efforts.  At  first  it  was  thought 
that  the  program  is  responsible  for  $47.5  million  in  savings  for  the  government  till  June  2006 
(Vitasek,  Geary,  Cothran  and  Rutner,  2006,  p.9).  According  to  Vitasek,  Cothran  and  Rutner 
(2006,  p.5),  the  total  savings  achieved  is  $217.5  million.  This  savings  include  the  one,  gained 
from  reducing  government  Program  Office  manning  from  265  people  to  48  people,  by  means  of 
$90  million  savings.  Therefore  it  is  obvious  that  the  savings  also  occurred  even  in  human 
resources  area. 

E-8C  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS) 

JSTARS  is  an  airborne  command  post. 

“The  E-8C  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS)  is  an  airborne  battle 
management,  Command  and  Control  platform.  From  a  standoff  position,  the  modified 
707-  300  manned  by  a  joint  Army-Air  Force  crew,  detects,  locates,  tracks,  and  targets 
hostile  surface  movements,  communicating  real-time  information  through  secure  data 
links  to  Air  Force  and  Army  command  centers.”  (Performance  Based  Logistics:  A 
Program  Manager’s  Product  Support  Guide,  2005,  p.5-4) 

Northrop-Grumman  has  TSSR  for  sustainment  of  JSTARS  over  a  maximum  contract 
period  of  22  years.  This  is  also  one  of  the  Reduction  in  Total  Cost  (RTOC)  pilot  programs  that 
benefited  from  the  PBL  efforts  (Performance  Based  Logistics:  A  Program  Manager’s  Product 
Support  Guide,  2005).  Boehlc  (2003)  states  that  JSTARS  flew  100  percent  of  scheduled  missions 
over  Afghanistan,  realizing  $30.8  million  in  fiscal  years  2001  and  2002.  Goure  (2010)  supported 
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the  idea  with  more  recent  data;  in  2007  the  Air  Force  analyzed  the  business  case  for  its  PBA  and 

concluded  that  it  generated  savings  of  $38  million. 

Naval  Inventory  Control  Point  (NAVICP):  Auxiliary  Power  Unit  (APU) 

This  PBL  contract  is  the  first  public/private  partnership  in  naval  logistics.  In  addition  to 

that,  it  is  also  one  of  the  Reduction  in  Total  Cost  (RTOC)  pilot  programs  that  benefited  from  the 

PBL  efforts.  Honeywell  Corporation  is  the  Product  Support  Integrator  (PSI)  in  this  partnership. 

This  10  year  (5  year  base  with  5-1  year  renewal  options)  contract  was  signed  in  June  of  2000 

(Vitasek,  Cothran  and  Rutner,  2007).  The  APU/Total  Logistics  Support  (TLS)  PBL  contract 

shifts  total  management  responsibility  for  APU  types  used  on  the  F/A-18,  S-3,  C-2,  and  P-3 

aircraft,  including  all  peculiar  components  and  accessories  (Performance  Based  Logistics:  A 

Program  Manager’s  Product  Support  Guide,  2005).  PBL  Guide  states  that  Life  Cycle  Cost  (LCC) 

savings  should  exceed  $50  million  through  this  partnership.  According  to  Vitasek  et  al.  (2007); 

in  addition  to  many  performance  improvements  a  $70  million  savings  was  documented  for  this 

program  in  August  of  2005.  APU  Total  Lifecycle  Support  program  took  the  PBL  awards  at 

component  level  in  2005.  It  is  estimated  to  achieve  $70  million  savings  over  the  life  of  the  10- 

year  contract  (Vitasek,  et  al.,  2006).  Therefore  it  exceeded  these  expectations.  In  a  recent  study 

Gansler,  Lucyshyn,  Harrington  and  Corl  (2011)  stated  that  $35  million  in  total  cost  savings  and 

cost  avoidance  occurred  with  $8.5  million  in  annual  inventory  savings  in  FRC-East  APU  public- 

private  partnership.  As  seen  the  savings  is  not  only  occurred  in  maintenance  but  also  occurred  in 

inventory.  Gansler,  et  al.’s  (2011)  recent  study  includes  the  recent  BCA. 

“The  BCA  concluded  that  the  Navy  would  save  $13.98  million  over  10  years  by 
awarding  the  DVD/TLS  contract  to  Honeywell — which  was  revised  to  a  $34.8  million 
savings  when  additional  benefits  were  considered.  In  2007,  the  Navy  identified  that  the 
cost  savings  were  greater  than  $50  million.  . . .  NAVAIR  credited  the  TLS  partnership 
with  more  than  30  reliability  improvements  which  it  estimated  would  produce  upwards  of 
$50  million  in  cost  avoidance  and  savings.  . . .  The  FRC-East  APU  public-private 
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partnership  has  captured  a  total  of  $35  million  in  benefits  for  the  Navy  to  date.”  (Gansler 
et  ah,  2011,  p.31-42) 

Navy/GE  F404 

GE  F404  Aircraft  Engine  is  the  power  of  Navy’s  front  line  fighter  and  attack  aircraft,  the 

F/A-l  Hornet.  This  is  A  PBL  partnership  program  with  General  Electric  to  increase  the 

availability  of  F/A-l  8  aircraft  engines  for  4  Vi  years.  According  to  Vitasek  et  al.  (2007)  the 

success  is  huge  for  this  program;  while  improving  performance,  there  has  been  a  reduction  in  the 

total  cost  of  ownership  of  13.8%  by  means  an  impressive  $79M  and  partners  expect  more  than 

that  by  ongoing  program.  By  these  achievements  the  program  was  awarded  at  the  sub-system 

level  (Vitasek  et  al.,  2006).  While  stating  performance  improvements,  Goure  (2010)  states  that 

there  is  reducing  in  inventory  costs  (and  thereby  costs  to  the  Air  Force)  of  roughly  2 1  percent.  As 

seen  this  is  another  cost  effective  successful  PBL  program  of  DoD. 

Naval  Inventory  Control  Point:  Aircraft  Tires 

“The  Naval  Inventory  Control  Point  (NAVICP)  Aircraft  Tires  PBL  contract  transfers 
traditional  Department  of  Defense  (DoD)  inventory  management  functions  to  the 
contractor,  which  will  guarantee  a  level  of  tire  availability  versus  physical  inventory. 
Under  this  vehicle,  the  contractor  is  tasked  to  become  the  single  supply  chain  integrator 
for  Navy  aircraft  tires  and  is  responsible  for  requirements  forecasting,  inventory 
management,  retrograde  management,  stowage,  and  transportation.”  (Performance  Based 
Logistics:  A  Program  Manager’s  Product  Support  Guide,  2005,  p.5-6) 

In  July  of  the  year  2001  Michelin  Aircraft  Tire  Company,  LLC,  and  Lockheed  Martin 

agreed  on  a  PBL  with  NAVICP.  Michelin  became  Navy’s  agency  to  procure,  manage,  and 

supply  spare  parts  for  naval  aircraft,  submarines,  and  ships  worldwide,  is  responsible  for  more 

than  400,000  items  of  supply,  $27  billion  of  inventory,  and  $4.2  billion  in  annual  sales 

(Mahadevia,  Engel  and  Fowler,  2006,  p.30). 
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Seventy  five  percent  reductions  occurred  in 
retail-level  inventories  at  continental  United 
States  air  stations  as  shown  in  Figure  2. 


This  will  reduce  TOC  for  Navy  absolutely. 


According  to  Navy,  it  is  projected  that  there  would  be  significant  cost  savings  totaling 
more  than  $46  million  over  15  years  (Mahadevia,  et  ah,  2006;  Kratz  and  Buckingham,  2010b; 
Beilis,  2001).  In  addition  to  that,  Kratz  and  Buckingham  (2010b)  think  that  the  privatization 
effort  of  aircraft  tires  continues  to  save  money  on  costs  associated  with  not  only  acquisition  but 
also  storage,  maintenance,  and  disposal  by  placing  these  responsibilities  on  Michelin. 

C-l  7  Globemaster 

C- 17  is  a  large  military  transportation  aircraft  used  for  strategic  lifting.  Under  the  C-l 7 
PBL  agreement,  Boeing  is  the  prime  vendor.  Boeing  is  responsible  for  the  entire  aircraft.  Boeing 
subcontracts  depot  repairs  to  the  U.S.  Air  Force  at  Wamer-Robins,  engine  work  to  Pratt  & 
Whitney  and  components  to  other  firms.  According  to  Pat  Finneran,  president  of  support  systems 
at  Boeing  IDS;  $10  million  in  annual  costs  have  been  saved  and  $500  million  in  duplicate 
infrastructure  costs  were  avoided  by  the  PBL  approach  (Canaday,  2006,  p.2-3).  For  C-l 7  PBL 
program  Goure  (2009)  states  $562  million  cumulative  savings  to  the  government  in  the  first  ten 
years.  Goure  (2011b)  also  used  C-l 7  example  in  his  recent  study  to  oppose  the  idea  of  going 
back  to  organic  maintenance,  due  to  the  over  $1  billion  savings  to  the  Air  Force.  He  states  that 
organic  structure  will  not  be  cost  effective. 


Figure  2.  Inventory  Reduction  in  A/C  Tires 
Program  (Mahadevia,  et  ah,  2006,  p.30). 
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Other  Examples 

Shadow  Unmanned  Aerial  Vehicle  (UAV):  UAS  PO  and  AAI  made  up  the  PBL  team, 
which  is  responsible  for  all  system.  This  team’s  goal  is  maintaining  the  warfighter  readiness 
with  increasing  cost  efficiency.  25  percent  contract  cost  reduction,  while  the  System  Status 
Readiness  (SSR)  rate  remained  consistently  above  90  percent  has  been  achieved  by  this  program 
(Owings,  2010,  p.16).  According  to  GOA  report  (2008),  the  maximum  amounts  authorized  in  the 
annual  cost-reimbursable  PBL  contract  for  the  support  of  this  system  were  reduced  by  28  percent 
from  fiscal  years  2006  through  2007. 

F-22  Raptor:  This  PBL  program  had  2008  PBL  System  Level  Award.  Lockheed  Martin 
is  responsible  for  F-22  sustainment.  40  percent  reduction  is  reported  in  costs  by  means  of 
hundreds  of  millions  dollars  savings  with  improved  aircraft  availability  (Goure,  2009,  p.l; 

Goure,  2010,  p.7). 

Tornado  Support  in  United  Kingdom  (U.K.):  This  is  one  of  the  five  pilot  PBL 

programs  with  BAE  in  U.K.  At  one  of  the  Royal  Air  Force  base,  Tornado  radar  support  requires 

50  percent  less  inventory  now,  and  costs  are  down  by  45  percent;  secondary  power  systems  on 

the  aircraft  cost  23  percent  less.  These  are  concluding  30  percent  less  support  costs  while 

achieving  all  of  the  milestone  objectives.  In  addition  to  that  the  agency  aims  to  cut  costs  by  up  to 

50  percent  while  improving  availability  with  a  10  year  program  (Canaday,  2006,  p.2). 

T-45  Trainer:  Navy  renewed  the  contract  with  Rolls-Royce  for  support  of  the  engines 

with  an  expectation  of  $65  million  for  5  year  contract  period. 

“According  to  the  fonner  Commander,  Naval  Air  Systems  Command:  The  success  of 
Perfonnance  Based  Logistics  (PBL)  has  allowed  the  Naval  Aviation  Enterprise  (NAE)  to 
improve  support  to  the  warfighter  and  achieve  weapon  system  readiness  at  lower  life 
cycle  costs.”  (Goure,  2010,  p.5) 
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F/A-18E/F  Integrated  Readiness  Support  Teaming  (FIRST)  Program:  According  to 
Goure  (2010)  this  program  is  one  of  the  major  successes.  The  expected  cost  savings  is  $688 
million  over  the  life  of  the  program  based  on  the  business  case  analysis. 

KC-130  J:  This  is  a  NAVAIR-NSWC  Crane  Partnership  program.  Geary,  Koster, 
Randall  and  Haynie  (2010)  cite  Camacho  (2008)  and  state  that  this  PBL  sustainment  program 
saved  the  government  $42  million  by  reducing  the  cost  per  flight  hour  by  nearly  75  percent  from 
2005  to  2007. 

Forward  Looking  Infrared  (FLIR)  for  H-60:  Raytheon  has  built  to  FLIR  for  H-60 
helicopters  according  to  DoD’s  specs.  They  had  a  PBL  agreement  which  were  projected  to  have 
around  $3 1  million  cost  savings,  but  by  June  2008  it  has  been  estimated  to  exceed  $42  million 
(Vitasek  and  Geary,  2008,  p.63). 

High  Mobility  Artillery  Rocket  System  (HIMARS):  This  system’s  support 
responsibility  is  under  Lockheed  Martin.  According  to  the  Anny  study;  $412  million  cost 
savings  over  the  traditional  method  of  organic  only  support  is  expected  (Vitasek,  et  al.,  2007). 

Army’s  M-l  Abrams:  A  potential  cost  reduction  of  $17  billion  in  operations  and 
sustainment  over  the  next  30  years  of  service  is  expected  from  Army’s  M-l  Abrams  PBL 
partnership  projects  (Boehk,  2003). 

JAVELIN:  This  is  an  anti-tank  weapon  to  which  Lockheed  Martin  supports  via  a  PBL 
agreement.  According  to  Spring  (2010);  industry  assesses  the  cost-benefit  improvement  derived 
from  the  program  to  be  10  percent. 

E-3D  Sentry  Airborne  Warning  and  Control  System  (AW ACS):  Sols  and  Verma 
(2007)  cite  Kemp  (2005)  and  state  AWACS  as  an  example  of  the  reduction  in  costs  that  will 
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result  from  higher  efficiencies  along  with  achieved  pre-set  performance  goals:  The  Royal  Air 

Force  in  the  U.K.  is  expecting  to  reduce  costs  by  12%  in  the  outsourced  maintenance  of  its  fleet. 

As  seen  from  the  examples;  PBL  is  an  effective  method  to  reduce  cost,  especially  in  an 

increasing  age/maintenance  cost  environment.  Under  Secretary  of  Defense  Jacques  Gansler 

described  this  phenomenon  with  ‘death  spiral’  term.  It  is: 

“. . .  a  cycle  where  aging  weapons  and  inefficiencies  in  the  support  programs  for  existing 
weapons  divert  defense  dollars  to  the  maintenance  accounts,  which  results  in  deferral  of 
the  procurement  of  new  weapons,  which  in  turn  results  in  older  and  more  expensive-to- 
maintain  weapons.”  (Gansler,  et  ah,  2011,  p.8;  Spring,  2010,  p.4) 

Therefore  it  can  be  simply  stated  that  PBL  is  invented  to  attack  death  spiral.  Gansler’ s 

aggressive  acquisition  reform’s  key  initiatives  include  increased  use  of  commercial  items, 

evolutionary  acquisition,  streamlined  acquisition  documentation,  and  performance  based 

logistics  (Kratz  and  Buckingham,  2010a).  It  is  obvious;  using  PBL  will  save  money.  What 

should  be  done  with  this  money  is  described  by  Spring  (2010,  p.4): 

“If  the  maintenance  costs  incurred  for  the  support  of  existing  weapons  can  be  reduced, 
the  savings  can  be  channeled  into  the  acquisition  of  new  weapons  that  will  be  less 
expensive  to  maintain.  While  this  step  will  require  larger  overall  defense  budgets  for  the 
core  defense  program,  it  can  help  to  initiate  a  reverse  dynamic  that  will  permit  the  death 
spiral  to  become  a  recovery  spiral.” 

Although  Gansler  is  seemed  to  be  an  aggressive  supporter  of  PBL  as  an  cost  effective 

tool,  Cebeci  (2009)  cites  Gansler  (2000)  when  he  states  “Performance-based  practices  are 

expected  to  help  the  Department  of  Defense  (DoD)  improve  performance,  encourage  innovation, 

and  increase  competition  in  services,  often  at  a  reduced  cost  to  the  government.”  He  states  that 

PBL  does  not  always  result  with  reduced  costs.  But  mostly  it  is  the  intent  for  PBL. 

“The  evidence  is  clear:  PBL  works.  PBL  delivers  dramatic  improvements  in  performance 
with  lower  operating  costs  across  the  total  life  cycle.  PBL  does  more  for  the  warfighter 
with  less  from  the  taxpayer.  Instead  of  paying  for  transactional  activities,  the  government 
and  industry  partners  deliver  improved  performance  at  lower  costs.”  (Fowler,  2009,  p.7) 
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According  to  Geary  and  Vitasek  (2005,  p.l) 


“PBL  is  the  most  impressive  thought  we’ve  seen,  with  both  intellectual  rigor  and  real 
results,  endeavoring  to  refonn  weapons  program  spending.  It’s  a  real  solution  to  a  real 
problem,  already  delivering  improved  perfonnance  at  lower  total  cost.” 

Hedden  (2005)  defines  PBL  and  counts  decreasing  costs  per  unit  usage  as  one  of  the  goals  of 

PBL. 


According  to  Vitasek  and  Geary  (2008,  p.65)  “PBL  has  been  proven  to  be  effective  as  a 

means  of  purchasing  higher  quality  results  at  lower  cost  to  the  government  by  allowing  defense 

contractors  to  utilize  industry  best  practices.  Facts  show  that  the  use  of  PBL  can  bring  extensive 

improvements  to  a  weapon  system,  while  lowering  costs  and  increasing  reliability  and 

availability.”  As  a  result  PBL  is  an  acquisition  method,  which  buys  pre-set  perfonnance 

outcomes  (or  even  improve  them)  with  lower  TOC. 

2.  Presence,  Determination  of  Metrics,  Performance  Outcomes 

In  fifty  five  out  of  eighty  one  documents,  metrics  or  perfonnance  outcomes  is  mentioned 

regarding  PBL  and  it  is  stated  as  an  obligation  for  it.  When  using  PBL  as  an  acquisition  method, 

perfonnance  outcomes  are  tried  to  be  bought.  Therefore  we  should  have  some  metrics  in  hand. 

“A  key  component  of  any  PBL  implementation  is  the  establishment  of  metrics.  Since  the 
purpose  of  PBL  is  ‘buying  perfonnance,’  what  constitutes  performance  must  be  defined 
in  a  manner  in  which  the  achievement  of  performance  can  be  tracked,  measured,  and 
assessed.”  (Performance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide, 
2005,  p.2-4) 

According  to  Federal  Acquisition  Regulations  (FAR)  subpart  37.601;  performance  based 
contracts  for  services  shall  include  perfonnance  work  statements  (PWS)  and  measurable 
perfonnance  standards  in  tenns  of  quality,  timeliness,  quantity,  etc.  (FAR  37.6;  Memorandum: 
Perfonnance-Based  Services  Acquisition,  2000).  Further  defines  PWS;  describing  the  required 
results  in  clear,  specific,  and  objective  tenns  with  measurable  outcomes  (GAO  Report,  2008). 
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DoD  PBL  guide  states  that;  a  successful  PBL  depends  on  these  metrics  which  accurately  reflect 
the  customers’  needs  and  can  be  an  effective  measure  of  PSI’s  performance.  In  PBL 
environment,  contract  requirements  shall  be  stated  in  perfonnance  terms,  limiting  the  use  of 
military  specifications  and  standards  to  Government-unique  requirements  only  (DoDD  5000.1, 
2003,  p.7).  DoD  also  issued  another  guidance  regarding  life  cycle  management  that  focuses  on 
life  cycle  metrics,  aligning  resources  and  readiness,  and  implementing  performance-based  life 
cycle  product  support  (Kratz  and  Buckingham,  2010a). 

Cebeci  (2009,  p.  1 5)  cites  these  regulations  and  also  Garrett  (2005).  Garrett  (2005)  states 
that  experts  in  US  federal  government  contracting  and  acquisition  management  think  that;  the 
essential  elements  of  PBA  are  performance  requirements,  perfonnance  standards,  perfonnance 
measures  and  metrics,  contractual  incentives,  and  appropriately  tailored  terms  and  conditions 
that  are  specific  to  the  unique  acquisition  environment.  In  addition,  he  found  in  his  study  that 
“Interviewees  acknowledged  that  while  it  is  difficult  to  define  the  best,  most  fair,  and  most 
reasonable  perfonnance  metrics  for  both  the  government  and  the  contractor,  defining  those 
standards  is  the  key  to  acquisition  success.”  (p.47)  Heller  (2001,  p.2)  states  that  Perfonnance 
Based  Logistics  consists  of  output  perfonnance  parameters  to  ensure  the  system  ready  capability. 
According  to  Kobren  (2009,  p.261)  the  key  in  PBL  is  to  establish  solid,  well  crafted,  integrated 
metrics  and  incentives  emphasizing  the  desired  perfonnance  outcomes. 

David  V.  Pauling,  Deputy  Undersecretary  of  Defense,  Materiel  Readiness,  and 
Maintenance  Policy,  admits  at  an  interview  with  AAD  in  December  2006  that  they  need  to  build 
on  lessons  learned  from  continuous  process  improvements,  and  apply  an  overall  perfonnance- 
based  outcomes  approach,  tying  all  budgeting  and  contracting  to  a  warfighter-supported  metric 
(Blumberg,  2007a,  p.42). 
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DoD  PBL  Guide  helps  to  find  out  what  these  metrics  should  be.  The  guide  cites  the 
memorandum  of  Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics 
Memorandum,  August  16,  2004,  Performance  Based  Logistics:  Purchasing  Using  Performance 
Based  Criteria)  and  determines  the  five  top-level  metrics;  operational  availability,  operational 
reliability,  cost  per  unit  usage,  logistics  footprint  and  logistics  response  time.  Therefore  “PBL 
outcomes  are  expressed  in  terms  of  operational  availability,  such  as  the  number  of  available 
flight  hours  for  an  aircraft.  Other  measures  are  increased  mission  reliability,  and  reductions  in 
operations  and  support  costs,  logistic  footprint,  and  logistics  response  time.”  (Blumberg,  2006a, 
p.57)  All  desired  outcomes  should  be  linked  to  these  metrics,  even  if  PSI  is  organic.  “For  all  PBL 
contracts,  warfighter  requirement(s)  will  be  linked  to  metrics  and  metrics  to  contract  incentives. 
For  all  organic  PBL  PSIs,  warfighter  requirement(s)  will  be  linked  to  metrics  and  metrics  to 
PBAs  between  the  PM  and  the  organic  PSIs.”  (Performance  Based  Logistics:  A  Program 
Manager’s  Product  Support  Guide,  2005,  p.3-30)  DoD  mandates  to  measure  support 
perfonnance  based  on  the  high-level  metrics,  such  as  availability  of  mission-capable  systems, 
instead  of  on  distinct  elements  such  as  parts,  maintenance,  and  data  (DoDD  5000.02,  2002, 
p.36).  Therefore  these  key  performance  parameters  are  so  important  to  the  warfighter  and  need 
identification  from  the  start  point  (Pettingill,  et  ah,  2004).  Hedden  (2005)  accepted  these  metrics 
as  PBL  goals. 

Paying  for  results  is  PBL’s  purpose.  Mitchell  (2008)  cites  Sandborn  (n.d.):  “The  military 
is  going  to  this  . . .  They’re  not  going  to  own  airplanes  anymore.  They’re  simply  going  to  buy 
availability  of  airplanes.”  PBL  can  be  simply  defined  as  buying  perfonnance,  support  outcomes 
at  a  whole  new  level,  (Vitasek,  et  al.  2006;  Venema,  2007;  Hedden,  2007)  instead  of  buying 
basic  elements  of  support  such  as  parts,  spares;  transaction  goods  or  services  (Geary  and  Vitasek, 
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2005;  Wuchenich,  2008;  Cothran,  2008;  Ott,  2008;  Goure,  2010)  and  besides  Vitasek  et  al. 
(2006,  p.2)  match  these  key  metrics  to  corresponding  questions: 


Table  3.  Vitasek  et  al.’s  questions  to  key  metrics. 


Metric 

Proble  m  Addre  s  s  e  d 

Operational  Availability  (Ao) 

Are  we  ready? 

Mission  Reliability 

Will  we  be  effective? 

Cost  per  Unit  Usage 

What  is  the  cost? 

Footprint 

How  much  “stuff’  do  we  need? 

Logistics  Response  Time 

Are  we  sustainable? 

Two  of  the  fifteen  attributes  of  a  good  PBL,  which  have  been  identified  by  as  part  of  the 

research  at  the  University  of  Tennessee,  are  below: 

“Performance  Definition:  Top-level  broad  scope  outcomes  optimizing  readiness, 
availability,  reliability,  cycle  time,  and  affordability. 

Performance  Measurement:  Minimum  number  of  top-level  ‘Outcome’  metrics  that 
measure  and  are  aligned  to  warfighter  needs  and  tightly  aligned  with  Support  Provider 
scope  of  authority.”  (Vitasek  and  Geary,  2008,  p.63) 

Harada  (2010)  categorizes  the  metrics,  can  be  used  in  PBL,  into  3  groups;  Effectiveness, 
efficiency  and  response  time.  In  his  categorization  effectiveness  includes  2  of  the  5  top-level 
metrics;  availability  or  readiness  (combined  regarding  top-level  metrics)  and  reliability.  He  also 
puts  customer  feelings  in  this  category.  He  made  a  detailed  description  for  each  group,  subgroup 
and  items  underneath.  Table  6  shows  the  metrics  he  came  up  with. 

Gansler  et  al.  (2011,  p.9),  Harada  (2010,  p.  17-19),  Bozkurt  and  Guducu  (2005,  p.  16-17) 
described  these  five  top-level  metrics: 

•  Operational  availability:  The  percent  of  time  that  a  system  is  able  to  conduct  operations  per 
demand. 

•  Operational  reliability:  The  measure  of  a  system  in  meeting  mission  success  objectives  (such  as 
a  sortie,  tour,  launch,  destination  reached,  capability,  etc.). 
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•  Cost  per  unit  usage:  The  total  operating  costs  per  the  appropriate  unit  of  measurement  for  a 
given  system  (such  as  costs  per  flight  hour,  steaming  hour,  launch,  and  mile  driven,  etc.). 

•  Logistics  footprint:  The  size  of  logistics  support,  adequate  to  deploy,  sustain,  and  move  a 
system  (such  as  inventory/equipment,  personnel,  facilities,  transportation  assets,  and  real  estate). 

•  Logistics  response  time:  The  period  of  time  from  logistics  demand  signal  sent,  till  it  has  been 
satisfied. 

Figure  3  shows  Harada’s  (2010)  metrics  and  categorization. 


Figure  3.  Harada  (2010)’s  Metrics  and  Categorization. 


Just  using  the  appropriate  metrics  in  contracts  is  not  enough.  There  are  other 
requirements  for  a  good  PBL.  Harada  (2010,  p.20)  cites  Sols,  Nowicki  &  Verma  (2008)  and 
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makes  suggestions  what  to  do  with  metrics:  Select  performance  metrics;  establish  baseline  values 
for  each  performance  metric;  set  target  values  for  each  perfonnance  metric,  define  an  n- 
dimensional  reward  scheme  and  measure  the  actual  effectiveness.  Berkowitz  et  al.  (2003) 
emphasizes  defining  and  clarifying  perfonnance  metrics  and  states  that  these  metrics  should  be 
easily  understood  and  quantified. 

According  to  Berkowitz  et  al.  (2003)  performance  levels,  metrics  should  be  aligned  to 

goals. 


DoD  PBL  Guide 
used  TOW  IT  AS  PBL 
program  as  a  good 
example  for  metrics 
determination.  In  that 
program  profitability  is 
linked  to  availability; 
therefore  higher 
availability  comes  with 


TOW-ITAS  PBL  Contract  Metrics 

Fixed  Price-  Performance  Adjustment 


•  5  Yr  Multiple-Year 
Contract 

•  CLS- 00-01 

•  PBL -02-06 

•  Maximum  Flexibility 

•  Separate  CLINs 

-  Transition 

-  War 


100% 

JZ  95% 


Operational  Readiness  =  Profit 

Profit  Plus  Maximum 
Performance  Adjustment 

+  $ 


Additional  Metrics  to  Prevent 
“Gaming" 

Separate  Metrics  for 
Contingency  Options 
Free  Issue  of  Repair  Parts  to 
Units 

Obsolescence  and  Tech  Refresh 
Contract  Contains  Plan  for 
Transition 


Profit  Plus  Minimum  Performance 
Adjustment 


'J0% 

85% 


Fixed  Price  Portion  of  Contract 


sss, 


pr  80% 


Minimum  Profit 


$s$ 


Contract  Price  Price  per  System  per  Year  “ 


Figure  4.  TOW  ITAS  PBL  Contract  Metrics 
(DoD  PBL  Guide,  2005,  p.3-23). 


higher  profits.  98-100  percent 
operational  availability  has  been  captured  by  Army  since  February  2001.  Figure  4  shows  this 
program’s  contract  metrics. 

Another  good  example  is  Shadow  UAV  PBL  program.  This  PBL  program  has  exceeded 
all  of  its  performance  goals  in  6  months  period  at  Operation  Iraqi  Freedom.  Figure  5  summarizes 
the  program  and  metrics  pyramid. 
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Shadow  200  PBL  Contract  Performance  Metrics: 


PArformanco  Moines  aro  woightod  as  follows: 

SSR  50% 

CWT  25% 

FSR  20% 

LMR  5% 

fSSfl  x  50)  .  | CWT  «  25)  ♦  (FSfl  x  20)  ♦  (IMH  x 


Metrie/Requlrement 


6  Mo  Performance 
(Including  OIF) 


05%  or  Higher  System  Status  35  2% 

Readiness  (SSR) 

SO\  or  Higher  Customer  WMt  Time 

(CWT)  93.1% 

3-12/1  or  Higher  Logistics 

Maintenance  Hello  7.9  10  1 
(LMR) 

Satisfactory  Field  Service 

Representative  Outstanding 

Performance  (FSR) 


Performance  Metrics  Pyramid 


UAV  Performance  Based  Product  Support  Contract  Strategy 
AN  EVOLUTIONARY  PROCESS 
May  2003  -  Sep  2004 

Product  Support  I  -  Cost  Plus  Fixed  Fee 


The  Coal  Plus  phases  provide  the  opportunity 
to  evaluate  true  cost  and  to  determine  the 
right  incentives  to  support  the  Fixed  Price 
phase. 

Provides  the  time  to  validate  and  verify  the 
metrics  and  Data  Collection  processes. 

The  key  Is  the  data  collection  and 


L  Implementation  during  Phase  I 


Better  System  Operational 

Readlness/Availabflity 

Increased  MTBSA 

Reduced  logistics  footprint 

Higher  overall  system  readiness  levels 


Full  PBL  Implementation  during  Phase  III 


I 

// 

°"\ 

wr 

FSB 

/http 

MTBF 

I1TBEFF 

KPP\ 

Logistics  Performance  Pyramid 


Figure  5.  Shadow  UAV  PBL  Program.  (DoD  PBL  Guide,  2005,  p.3-25). 

One  of  the  lessons  learned  in  JSTARS  PBL  program’s  success  comes  from  focusing  on 
the  warfighter  requirements  and  managing  it  effectively  with  an  extensive  set  of  metrics 
(Lessons  Learned,  2004,  p.  2).  The  PSI,  Northrop  Grumman,  is  awarded  according  to  both 
qualitative  and  quantitative  metrics.  Customer  feedbacks  fonn  the  qualitative  part  and  depot 
delivery  of  aircraft,  non-mission  capable  for  supply  rates,  mission  capable  delivery  times,  in¬ 
flight  trainer  sorties,  readiness  spares  package  fill  rate,  and  contract  cost  performance  form  the 
quantitative  part.  But  there  are  issues  about  measuring  aircraft  availability  or  logistics  footprints 
due  to  having  metrics  to  measure  neither  of  them  (Pettingill,  et  ah,  2004,  p.18-23). 

Like  the  Air  Force,  the  Navy  has  lots  of  experience  and  suggestions  about  detennining 
and  using  these  metrics.  It  is  advised  by  Navy  that  “sound,  realistic  performance  metrics”  should 
be  developed  for  success.  One  of  the  other  concerns  of  Navy  is  “early  establishment  and  use  of 
metrics”  (Lessons  Learned,  2004,  p.  2). 

Global  Hawk  PBL  program  with  Northrop  Grumman  is  a  good  example  for  early 
implementation  of  metrics.  Figure  6  shows  the  roadmap.  This  program  is  considered  best 
practice  of  PBL  implementation  scheduling. 
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“The  schedule  is  particularly  noteworthy  because  it  presents  objectives  for  the  integration 
of  PBL  throughout  an  8-year  period  (FY04  -  FY1 1).  ...  PM  started  considering  PBL  as  a 
support  strategy  early  in  FY04  by  defining  the  metrics,  costs,  and  the  systems  used  to 
collect  this  data.”  (Pettingill,  et  ah,  2004,  p.20) 


FY04 

FY05 

FY06 

FY07 

FY08 

FY09 

FY10 

FY11 

Define 

Metrics 

Cost 

ID  Dat; 
System 

PBL 

Sustainment 


r 

rXORE 

Decisior 

SORAP  2 

SORAP  3 

Develop  Integrated 
Database  apd  Decision 
Support  Tools 


Stand-Up 

JCAPS 


Establish  Performance 
Baseline 

Measured  by  JCAPS 


Initiate 

Tailored 

LSA 


Full  JCAPS 
Implementation 


Stand-up 
|  Spares 
Modeling 


Figure  6.  Global  Hawk  Sustainment  Roadmap.  (Pettingill,  et  al,  2004,  p.21). 


In  F-22A  Raptor  program,  metrics  took  two  years  to  mature  at  the  beginning  of  the 
program  (Burkett,  2008,  p.3).  These  are  examples  of  ‘the  earlier  the  negotiations  begin  the  better 
metrics  we  came  up  with.’ 

As  experienced  by  Navy,  in  H-60  FLIR  program  the  contract’s  performance  metrics 
should  be  aligned  to  end-user  applicable  metrics  (Lessons  Learned,  2004,  p.  3). 

The  F- 117  PBL  program  which  is  established  between  AF  and  Lockheed  Martin  has  “7 
top  level  objective  metrics  that  optimize  system  availability  and  readiness,  as  well  as  4  subjective 
metrics  that  pertain  to  overall  support  management  efficiency”  under  award  portion  (Vitasek,  et 
al.,  2006,  p.5).  Lockheed  Martin  took  over  the  entire  scope  of  support  for  the  aircraft,  with 
operational  readiness  targets  as  contract  success  measures  (Blumberg,  2006a,  p.58).  This 
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program  is  considered  a  metrics’  best  practice,  due  to  the  effort  of  weighting  and  refining  the 
most  important  perfonnance  metrics.  On  the  other  hand  some  of  the  metrics  used  in  this  program 
cannot  be  identified  as  linked  with  desired  outcomes  for  overall  aircraft  availability  or  other  DoD 
PBL  outcomes,  “i.e.,  system  reliability,  total  operating  costs,  logistics  response  time,  and 
logistics  footprint.”  (Pettingill,  et  al,  2004,  p.15-23)  Although  this  is  a  ‘lessons  learned’  issue, 
this  program  won  a  DOD  PBL  award  for  the  most  successful  systems  level  PBL  in  2005. 

Boeing  is  contracted  as  C-17  nose  to  tail  supporter  for  Air  Force.  Sustainment 
partnership,  aircraft  availability  performance  metric,  customer  satisfaction  ratings,  and  target 
cost  are  aligned  with  profit  of  PSI.  “For  example,  Boeing  cannot  overstock  its  logistics  pipeline 
to  ensure  that  aircraft  remain  mission  capable  without  considering  inventory  procurement  and 
carrying  costs.”  This  program  is  considered  as  the  best  example  of  a  contract,  tying  perfonnance 
to  an  overall  availability  metrics  according  to  Pettingill,  et  al.  (2004,  p.18). 

In  C-130J  Sustainment  PBL  program  stockage  effectiveness,  customer  wait  time,  mission 
capability  hours  are  measured  as  the  supply  chain  metrics.  Sustaining  engineering  response  time 
is  also  measured  to  track  the  timeliness  of  the  response.  And  upon  all  stands  the  customer 
satisfaction.  Mission  capability  is  not  a  metrics,  because  this  partnership  is  not  responsible  for 
the  whole  nose-to-tail  supply  chain  (Blumberg,  2007b,  p.6).  Therefore  responsibilities  are  set 
clearly.  In  addition,  the  metrics  are  planned  to  be  changed  by  timeline  according  to  customer 
needs.  PSI  is  “incentivized  sustaining  engineering  and  customer  satisfaction  in  Year  One.  Supply 
chain  efficiency  metrics  will  be  phased  in  at  the  beginning  of  Year  Two.”  (Blumberg,  2007b, 

p.6) 

Sols  and  Venna  (2007,  p.40-46)  give  an  example  from  business  life;  the  Spanish  railway 
state  company  RENFE  has  contracted  CAF,  for  the  support  of  regional  TRD  trains,  on  certain 
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operational  performance  measures,  such  as  availability  and  mission  reliability.  But  due  to  “ill 
definition  of  the  effectiveness  metrics  and  of  the  domain  of  responsibility  of  each  party”  conflicts 
have  been  experienced.  It  is  an  obligation;  desired  outcomes  should  be  determined  appropriately 
and  do  not  let  any  misunderstandings.  In  this  PBL  program  they  used  “operational  availability 
(the  trains  are  to  be  ready  for  departure  at  specified  times)  and  mission  reliability  (the  trains 
should  arrive  at  their  destination  stations  on  time,  provided  they  departed  from  the  origin  stations 
on  time)”  In  some  cases  trains  experienced  delays  which  are  not  caused  by  contractor’s  fault  but 
decrease  performance  by  means  profit  and  caused  conflicts. 

They  also  give  a  good  PBL  example  regarding  metrics  about  C-212  and  CN-235  in  South 
America;  PBL  is  arranged  regarding  certain  fleet  availability.  Aircraft  status  has  been  set  as: 

“-  Operational  (ready  to  fly) 

-  Under  planned  maintenance  (not  ready  to  fly,  but  its  unavailability  is  accepted  due  to 
already  programmed  preventive  maintenance  actions) 

-  Aircraft  on  ground  (aircraft  not  available  due  to  failure  of  systems  or  equipment  that 
need  to  be  repaired;  the  contractor  is  responsible) 

-  Under  maintenance  (aircraft  not  available  due  to  failures  induced  by  the  system  user; 
client  responsible).”  (Sols  and  Venna,  2007,  p.44) 

Vitasek  and  Geary  (2007,  p.4-5)  use  a  PBL  program  example  from  software  industry.  In 
a  traditional  acquisition  approach,  customer  and  supplier  suffers  from  outsized  inventory  costs. 

In  PBL  environment  and  especially  where  understanding  and  “agreement  about  the  software 
giant’s  desired  outcomes”  are  provided;  supplier  utilizes  a  better  capacity,  customer’s  service 
level,  satisfaction,  fill  rate  goes  up,  and  inventory  and  obsolescence  goes  down. 

Vitasek  (2007)  uses  ‘aircraft  tire  PBL  program  with  Michelin’  when  describing  the  need 
of  aligning  desired  outcomes  with  metrics  in  the  conference. 

In  Blumberg’s  (2007a,  p.44)  study,  Vitasek  states  that  using  many  metrics  will  make  you 
lose  focus,  emphasis;  but  using  five  or  less  will  show  progress  to  meet  objectives.  According  to 


28 


her  one  of  the  success  key  is  having  few  metrics;  5  or  less  (Blumberg,  2007b;  Vitasek  and  Geary, 

2007).  Steven  Reid  from  AAI  (Shadow  UAV  PBL  program’s  PSI)  acknowledges  this  statement 

due  to  the  achievements  of  meeting/exceeding  availability  goals.  Vitasek  (2007)  used  C17 

sustainment  PBL  program  (about  5  metrics)  and  radar  sustainment  program  with  Rapheon  (1 

metrics;  operational  availability)  as  good  examples  of  using  few  metrics  and  stated  that  the  less 

number  and  the  ‘more  broad  you  described  the  metrics’  is  better. 

Devries  (2004)  finds  that  perfonnance  based  metrics  are  enabler  and  have  a  positive 

effect  in  implementation  of  PBL.  To  use  this  enabler  effectively  partners  should  work  closely 

and  link  those  to  desired  outcomes.  Although  it  is  one  of  the  biggest  five  challenges  identified  by 

the  PBL  Survey  respondents,  according  to  Newsome  (2008,  p.4),  customer  requirements  are  the 

most  important  thing  to  be  detennined  clearly,  without  any  misunderstanding. 

Literature  suggests  using  high/top-level  metrics;  operational  availability,  operational 

reliability,  cost  per  unit  usage,  logistics  footprint  and  logistics  response  time  and  using  them  as 

broad  as  it  can  be.  Also  Goure  (2010)  supports  that  and  adds,  two  of  the  factors  in  Navy’s 

success  are  focusing  on  availability,  reliability,  obsolescence  and  cost;  and  the  establishment  of 

specific,  appropriate  perfonnance  metrics.  In  the  simplest  way  of  using  metrics  according  to  the 

level  of  PBL,  as  Canaday  (2006,  p.l)  describes: 

“Level  1:  Components,  for  example  aircraft  tires.  Perfonnance  is  the  consistent  and 
timely  delivery  of  needed  components. 

Level  2:  Major  subsystems,  such  as  aircraft  engines.  Availability  is  the  main  perfonnance 
metric  here. 

Level  3:  Entire  aircraft.  Availability  is  the  goal. 

Level  4:  Mission  Capability.  Defense  officials  eventually  may  pay  private  finns  for 
providing  both  readiness  and  steady  enhancements  in  capabilities  for  military  aircraft.” 

As  a  result  PBL  is  buying  the  desired  outcomes  which  should  be  detennined  clearly  after 

detailed  negotiations.  According  to  Sols  and  Verma  (2007,  p.40); 
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.  in  order  for  a  PBL  contract  to  yield  the  desired  results  it  is  essential  to  have  a 
thorough  agreement  on  the  metrics  to  be  used  to  represent  system  effectiveness,  as  well 
as  an  agreed  reward  scheme  that  links  reward  to  achieved  effectiveness.” 

PBL  is  moving  from  transactional  through  operational  acquisition.  Fowler  (2010,  p.  1 8)  states 

that: 


“. .  .Product  Support  for  the  2 1st  Century,  the  DoD  strategy  for  product  support  has  been 
evolving  from  traditional  transactional  logistics  concepts —  in  which  the  components  of 
readiness  are  acquired  as  discrete  unit  transactions — to  a  stronger  emphasis  on  acquiring 
the  operational  readiness  outcomes  themselves.” 

Consequently  well-defined  metrics  is  one  of  the  keys  for  PBL’s  success. 

3.  Performance  Improvement 

In  fifty  two  out  of  eighty  one  documents,  perfonnance  improvement  is  mentioned 
regarding  PBL  with  an  emphasis  on  it.  DoD’s  one  of  the  intents  in  inventing  PBL  is  maximizing, 
improving,  optimizing  performance.  According  to  Gansler  et  al.  (2011);  PBL  is  a  useful  tool 
against  ‘death  spiral’.  In  addition  to  cost  efficiency,  operational  perfonnance  became  one  of  the 
most  important  metrics  of  DOD’s  acquisition  programs  especially  in  areas  of  support  and 
sustainment  (Owings,  2010). 

PBL  is  mandated  as  an  acquisition  method  whenever  feasible  to  maximize  performance 
(Memorandum:  Performance-Based  Services  Acquisition,  2000;  Blumberg,  2006a;  Fowler, 
2009).  “The  primary  objective  of  Defense  acquisition  is  to  acquire  quality  products  that  satisfy 
user  needs  with  measurable  improvements  to  mission  capability  and  operational  support,  in  a 
timely  manner,  and  at  a  fair  and  reasonable  price.”  (DoDD  5000.01,  2003,  p.3;  Pettingill,  et  al., 
2004,  p.iv)  DoD  shows  adequate  emphasis  on  ‘optimizing  total  system  availability’  as  a  PBL 
goal  in  its  definition  (Pettingill  et  al.,  2004,  p.l;  Hedden,  2005,  p.3;  Vitasek,  et  al.,  2006,  p.7; 
Kobren,  2010,  p.193). 
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PBL  is  a  transition  from  on  production  and  end-item  delivery  to  long-term  product 
availability  and  sustainability.  PBL  is  “maximizing  asset  uptime  or  operational  performance.” 
(Newsome,  2008,  p.2;  Kim,  Cohen  and  Netessine,  2006,  p.l)  For  the  win-win  solution  of  PBL; 
PSI  should  find  ways  to  engineer  improvements  (Geary  and  Vitasek,  2005;  Kratz  and 
Buckingham,  2010a). 

According  to  DoD  PBL  guide;  all  PBL  agreements  should  include  reliability  growth 
targets.  Sols  and  Verma  (2007)  states  that  while  supportability  aspects  are  considered  in 
contracts;  contractor  will  prepare  and  conduct  a  reliability  growth  program.  These  are  the  cases 
where  reliability  is  the  desired  outcome  of  customer.  One  of  the  key  elements  in  PBL 
environment  is  “continuous  improvement  of  weapon  system  supportability”,  availability  (Kratz, 
2001,  p.12;  Blumberg,  2006b,  p.58;  Vitasek  and  Geary,  2007,  p.l;  Goure,  2009,  p.l).  Mahadevia 
et  al.  (2006,  p.30)  state  that  demand  for  parts  is  reduced  “through  reliability  growth  and 
obsolescence  management.” 

In  DoD’s  Designing  and  Assessing  Supportability  Guide  (2003,  p.6)  System  Operational 
Effectiveness  (SOE)  is  described  as  “dependency  and  interplay  between  system  performance, 
availability  (reliability,  maintainability,  and  supportability),  process  efficiency  (system 
operations,  maintenance,  and  logistics  support),  and  system  life  cycle  cost.”  With  SOE  approach 
program  managers  (PM)  will  focus  on  “overall  objective  of  maximizing  the  operational 
effectiveness  of  weapon  systems”  (Designing  and  Assessing  Supportability  Guide,  2003,  p.6; 
Kratz,  2001,  p.12)  Improved  SOE  means  improved  performance,  means  enhance  system 
capability.  PBL  is  developed  and  implemented  by  PM  to  “optimize  total  system  availability 
while  minimizing  cost  and  logistics  footprint.”  (DoDD  5000.01,  2003,  p.1-1;  Kobren,  2010, 
p.187;  Goure,  2010,  p.2) 
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Mirazahosseinian  and  Piplani  (2011)  find  that  “to  achieve  a  desired  availability  level,  the 
supplier  has  to  improve  the  component  reliability  and  the  repair  time”.  Improvement  is  desired 
and  to  do  that  designing  should  be  taken  in  account.  The  underlying  principle  for  improvement  is 
redesigning.  ‘The  earliest  the  emphasis  on  engineering  design’  gives  the  better  improvement  in 
reliability,  maintainability  in  the  future  (Trovato,  2004).  “Emphasis  is  placed  on  designing  for 
increased  reliability  and  reduced  logistics  footprint  and  on  providing  for  effective  product 
support  through  performance-based  logistics  (PBL)  strategies.”  (DoD  Designing  and  Assessing 
Supportability  Guide,  2003,  p.3) 

Table  4.  Examples  used  in  documents  -  Performance  Improvement. 


GE  404 

APU 

F/A-18 

F-22 

A/C 

Tires 

F-117 

HI  MARS 

C-17 

H-60 

FUR 

AEGIS 

Software 

Co. 

C-130J 

UK 

CH47 

UK 

NIMROD 

UK 

TORNADO 

UK 

FV430 

Shadow 

UAV 

ITAS 

J  STARS 

BAE:  Guide  (2010) 

X 

Beilis  (2001) 

X 

Berkowitz  et  al. 
(2003) 

Blumberg  (2006a) 

X 

Blumberg  (2006b) 

X 

Blumberg  (2007b) 

X 

Boehk  (2003) 

X 

Canaday (2006) 

X 

X 

X 

DoD  PBL  Guide 
(2005) 

X 

Fowler  (2009) 

X 

X 

X 

X 

X 

Gansler,  Lucyshyn, 
Harrington  and 

Corl  (2011) 

X 

X 

X 

X 

X 

X 

X 

Geary  (2006) 

X 

X 

X 

X 

Goure  (2009) 

X 

Goure  (2010) 

X 

X 

X 

X 

X 

X 

X 

Goure  (2011b) 

X 

Mahadevia,  Engel 
and  Fowler  (2006) 

X 

Marietta  (2008) 

X 

Ott  (2008) 

X 

X 

Owings  (2010) 

X 

Sols  and  Verma 
(2007) 

Vitasek  and  Geary 
(2007) 

X 

Vitasek  and  Geary 
(2008) 

X 

Vitasek,  Cothran 
and  Rutner 
(2006) 

X 

X 

X 

X 

X 

Vitasek,  Cothran 
and  Rutner 
(2007) 

X 

X 

X 

Vitasek,  Geary, 
Cothran  and 

Rutner  (2006) 

X 

X 

X 

Total 

7 

6 

6 

5 

4 

4 

3 

3 

3 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 
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Twenty  four  documents  used  examples  to  describe  the  importance  of  performance 
improvement  in  PBL.  Table  4  shows  the  examples  used  by  corresponding  documents. 

Navy/GE  F404 

This  program  is  considered  to  be  a  huge  success.  This  program  achieved  92%  availability 
at  first  (Vitasek  et  al.,  2006,  p.4;  Goure,  2011,  p.7).  Fowler  (2009,  p.6)  and  Gansler  et  al.  (2011, 
p.  1 5)  declare  a  46%  availability  improvement.  In  the  first  2  years,  component  availability  goes 
up  to  99%  from  50%,  fleet  backorders  have  been  reduced  to  0  from  718,  and  repair  turnaround 
time  goes  down  to  47  from  120  days  and  the  life  expectancy  is  increased  by  20%  (Vitasek,  et  ah, 
2007,  p.22;  Vitasek,  et  ah,  2006,  p.8;  Ott,  2008,  p.61). 

NA  VICP:  APU 


“The  contract  provides  30  percent  to  60  percent  reliability  improvement  guarantees,  2- 
day  delivery  guarantees  for  high-priority  requirements,  obsolescence  management, 
product  support  engineering,  and  surge  capability  up  to  120  percent  of  annual  flight 
hours.  ...  awaiting  parts  reduced  from  232  to  0,  backorders  reduced  from  125  to  0,  supply 
materiel  availability  increased  from  65  percent  to  over  90  percent,  over  75  reliability 
improvements,  and  98  percent  of  requisitions  received  during  Operation  Enduring 
Freedom  filled  within  contractual  requirements  despite  a  60  percent  increase  in  demand.” 
(Perfonnance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide,  2005,  p.5- 
6) 


In  August  of  2005,  this  program  was  awarded  due  to  the  improvement  achievement  from 


65%  in  2000  to  97%  and  spares  inventory  levels 
decreased  25%  recently  (Vitasek,  et  ah,  2007,  p.21; 
Vitasek,  et  ah,  2006,  p.8). 

According  to  Beilis  (2001,  p.19)  material 
availability  for  APU  improved  to  the  target;  90% 
from  67  %  and  component  reliability  improvements 
are  as  good  as  material  availability  improvements. 


292% 


■  %  Improvement  in  Reliability 


Figure  7  APU  Component  Reliability 
(Beilis,  2001,  p.  19). 
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Figure  7  shows  component  reliability  improvements  for  APU. 

From  July  2000  to  October  2002,  the  number  of  APUs  awaiting  depot  repair  is  gone  from 
1 18  to  0,  backorders  from  125  to  26,  average  delivery  time  from  35  days  to  5.4  days  (Gansler,  et 
al,  2011,  p.75).  Recent  studies  show  that  “availability  is  now  95  percent.  Back  orders  have  been 
eliminated  and  wait  time  reduced  from  35  days  to  5.”  (Goure,  2011,  p.5;  Gansler,  et  al.,  2011, 
p.v) 

F/A-18E/F  Integrated  Readiness  Support  Teaming  (FIRST)  Program 

“The  mission  capable  rate  for  the  F/A-18E/F  improved  from  57  percent  in  2001  to  73 
percent  in  2007  and  the  program  increased  aircraft  availability  from  67  percent  to  85  percent.” 
(Goure,  2010,  p.5)  In  the  traditional  approach  the  overall  availability  rate  was  73%;  PBL 
achieved  a  rate  of  85%  eventually  (Sols  and  Verma,  2007,  p.47).  For  the  material  availability 
PBL  achieved  a  rate  of  85%  from  67%  (Vitasek,  et  al.,  2006,  p.2;  Geary,  2006,  p.78).  According 
to  Canaday  (2006,  p.3)  “. . .  time  on  repairs  has  been  cut  33  percent,  Hornet  readiness  is  the 
highest  among  the  U.S.  Navy's  tactical  aircraft,  mission-capable  rates  are  up  20  percent”.  Fowler 
(2009,  p.6)  and  Gansler  et  al.  (2011,  p.  1 5)  state  23%  availability  improvement  for  F/A-18E/F 
aircraft. 

F-22  Raptor 

This  program  was  awarded  in  2008  for  “for  producing  the  highest  readiness  rates  in  the 
program's  history.”  (Marietta,  2008,  p.l)  Fowler  (2009,  p.6)  and  Gansler  et  al.  (2011,  p.  1 5)  state 
15%  availability  improvement.  In  addition  these  mean  time  between  failures  (MTBF)  increased 
by  69%,  mission  rate  improved  by  15%,  repair  time  reduced  by  20%  (Marietta,  2008,  p.l;  Goure, 
2009,  p.  1).  In  a  more  recent  study  Goure  (2010, p. 7)  states  that  “aircraft  availability  improved  by 
nearly  20  percent.” 
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NA  VICP:  Aircraft  Tires 


By  this  PBL  program,  availability  requirements  are  met  over  98%  of  the  time  and 
delivery  time  is  1  day  in  average  where  at  most  2  days  is  required.  (Vitasek,  et  ah,  2006,  p.4) 
According  to  Mahadevia  et  al.  (2006,  p.31)  “on  time  tire  delivery  from  86  percent  to  96.4 
percent,  and  sustained  performance  remains  in  the  high  90s.”  Figure  8  shows  the  fill  rates  of  this 
program  by  time.  On  the  other  hand  backorders  are  gone,  “overall  tire  inventory  has  been 
reduced  by  66%  at  the  retail  level  and  92%  at  the  wholesale  level.”  (Vitasek,  et  al.,  2006,  p.4). 
Since  the  beginning  100%  fill  rate  is  sustained.  This  caused  75%  reduction  in  retail  level  of 
inventories  in  US  Navy  (Mahadevia  et  al.,  2006,  p.31).  Figure  9  shows  the  inventory  levels  at 
retail. 

Fowler  (2009,  p.6)  and  Gansler,  et  al.  (2011,  p.  15)  state  that  17%  availability 
improvement  is  experienced  by  this  program. 


Figure  8.  A/C  Tires  On  Time  Fill  Rate  Figure  9.  A/C  Tires  Inventory  Level  at 

(Mahadevia,  et  al.,  2006,  p.31).  Retail  (Mahadevia,  et  al.,  2006,  p.32). 

F-ll  7  Nighthawk  and  Engines 

Lockheed  Martin  has  achieved  improved  mission  readiness  in  the  F-l  17  by  reducing 
“response  time  from  80  hours  to  23.4  hours  and  engineering  disposition  response  time  from  190 
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hours  down  to  a  mere  2. 1  hours.  This  is  the  equivalent  of  the  Air  Force  having  one  additional 

aircraft  available  per  day.”  (Vitasek  et  ah,  2006,  p.5;  Bloomberg,  2006a,  p.59). 

For  engines,  Pratt  &  Whitney  agreed  to  keep  a  specified  number  of  engines  at  a  specified 

location  and  paid  for  engine  cycle  according  to  the  PBL  contract.  From  2000  to  2005,  85% 

increase  is  experienced  for  the  engines’  ‘time  on  wing’  (the  interval  between  service  events  that 

require  an  engine’s  removal  from  the  wing)  (Geary,  2006,  p.78).  According  to  Blumberg  (2006b, 

p.62)  achievements  are  spectacular  within  this  PBL  program: 

“A  P&W  spokesperson,  Heather  Summerer,  said  that  with  continuing  upgrades  under 
PBL  contracts,  the  time  on  wing  for  the  FI  17  engine  has  increased  215  percent  over  the 
past  8  years.  She  also  reported  that  turn  times  have  been  decreasing:  25  percent  for 
engines  and  43  percent  for  LRU  (line  replaceable  units).” 

Other  Examples 

High  Mobility  Artillery  Rocket  System  (HIMARS):  This  PBL  program  includes  3 
perfonnance  metrics.  In  a  short  period  PSI  met  these  metrics  with  99+%  System  Status 
Readiness  against  a  goal  of  92%;  a  mission  capable  turnaround  time  of  one  hours  against  a  goal 
of  96  hours,  and  a  repair  turnaround  time  of  two  days,  against  a  goal  of  five  days  (Vitasek,  et  ah, 
2007,  p.23;  Goure  2010,  p.6).  Ott  (2008,  p.  61)  supports  this  in  his  study  with  project  manager’s 
statement: 

“Lockheed  Martin  HIMARS  field  service  representatives  are  serving  in  Iraq  and 
Afghanistan.  ‘You  almost  can’t  increase  availability,’  says  Col.  David  Rice,  project 
manager  for  Precision  Firing  Rocket  and  Missile  Systems  in  the  Program  Executive 
Office  for  Missiles  &  Space  at  Huntsville,  Ala.  It  is  hitting  99.98%,  he  said.” 

C-17  Globemaster:  By  this  program  high  MICAP  rates  is  achieved  with  perfonnance 

extensions  (Geary,  2006,  p.78;  Goure  2010,  p.7).  Major  ‘aircraft  availability  improvement’  is 

experienced  (Goure,  2011b,  p.2). 
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Forward  Looking  Infrared  (FLIR)  for  H-60:  From  beginning  of  this  PBL  program  in 
October  2003,  100  percent  availability  rate,  40  percent  growth  in  system  reliability  improvement 
and  a  65  percent  repair  response  time  improvement  are  experienced  (Vitasek  and  Geary,  2008, 
p.63).  Fowler  (2009,  p.6)  and  Gansler,  et  al.  (2011,  p.  1 5)  state  14%  availability  improvement  in 
this  program. 

Aegis  Cruiser:  This  PBL  program  is  another  success.  Its  material  availability  increased 
from  62  %  to  94  %  (Geary,  2006,  p.78;  Vitasek,  et  al.,  2006,  p.2). 

Major  Software  Company: 

“In  the  software  business,  matching  supply  with  demand  is  notoriously  difficult.  Frequent 
product  revisions  make  the  job  even  tougher.  And  yet,  missing  a  peak-season  selling 
opportunity  or  product  launch  is  disastrous  because  the  margins  are  so  high.  A  typical 
product  costs  less  than  $5  to  make,  but  software  applications  can  sell  anywhere  from  $50 
to  several  hundred  dollars.”  (Vitasek,  et  al.,  2006,  p.8) 

The  firm  found  the  exit  way  in  PBL.  As  a  result  the  company  “maintained  a  98%  service 
level  rate  at  the  new  48-  hour  cycle  time,  and  cut  inventory  to  less  than  7%  —  a  whopping 
reduction  of  27%. ”  (Vitasek  and  Geary,  2007,  p.5;  Vitasek,  et  al.,  2006,  p.8) 

C-130  J:  According  to  Blumberg  (2007b,  p.8),  metrics  of  a  component  to  a  fleet  activity 
relative  to  the  engine  goes  up  to  95  %  from  43%,  turn  times  reduced  by  nearly  30%,  back  orders 
are  reduced  from  718  to  0  in  15  months. 

U.K.  CH-47  Chinook:  This  PBL  agreement  is  between  Boeing  and  UK  Ministry  of 
Defence.  “The  program  has  exceeded  all  expectations,  pushing  the  ‘mission  capable’  rating  to  90 
percent,  15  percent  higher  in  each  of  the  last  three  years  than  was  anticipated.”  (Gansler,  et  al., 
2011,  p.24) 

U.K.  Nimrod:  One  of  the  pilot  PBL  programs  in  U.K.  is  supporting  Nimrod  aircrafts.  By 
this  program  aircraft  availability  is  increased  by  8%  (Canaday,  2006,  p.2). 
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U.K.  Tornado:  “At  one  Royal  Air  Force  base,  CS&S  reduced  Tornado  downtime  by  35 
percent  and  man-hours  by  40  percent.  Tornado  radar  support  now  requires  50  percent  less 
inventory,  and  costs  are  down  45  percent.”  (Canaday,  2006,  p.2) 

U.K.  FV430  Armored  Personnel  Carriers:  “...British  Army  FV430  armoured 
personnel  carriers  received  a  new  engine  and  drivetrain  to  bring  them  up  to  ‘Bulldog’ 
specification.  The  upgrade  extends  the  useful  service  life  of  the  platform  by  20  years,  greatly 
increases  perfonnance  and  reliability  and  significantly  reduces  support  costs.”  (BAE:  Total 
Perfonnance,  2010,  p.8) 

TOW  Improved  Target  Acquisition  System  (ITAS):  “According  to  the  Army,  the  PBA 
to  support  the  improved  target  acquisition  system  (ITAS)  for  the  TOW  missile  has  provided 
better  than  90  percent  availability  over  the  past  seven  years”  (Goure,  2010,  p.7). 

Shadow  Unmanned  Aerial  Vehicle  (UAV):  According  to  Owings  (2010,  p.16)  system 
status  readiness  goes  up  to  90%  and  above  since  2008  and  reliability  growth  rate  duplicated  itself 
regarding  2006-2008  data  ,  depot  mean  down  time  is  less  than  60  days. 

E-8C  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS):  In  Afghanistan 
JSTARS  flew  all  missions  and  by  PBL  increased  its  availability  for  training  to  99  percent  from 
2001  to  2002  (Boehk,  2003,  p.l). 

As  seen  from  the  examples;  one  of  the  key  successes  of  PBL  is  improving  perfonnance. 

“PBL  has  been  proven  to  be  effective  as  a  means  of  purchasing  higher  quality  results  at 
lower  cost  to  the  government  by  allowing  defense  contractors  to  utilize  industry  best 
practices.  Facts  show  that  the  use  of  PBL  can  bring  extensive  improvements  to  a  weapon 
system,  while  lowering  costs  and  increasing  reliability  and  availability”  (Vitasek  and 
Geary,  2008,  p.65) 

Beilis  (2001,  p.19)  states  that  higher  material  availability,  less  delivery  times,  increased 
reliability,  timely  resolution  of  obsolescence  issues,  improved  readiness  make  PBL  a  way  of  life. 
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According  to  Bozkurt  and  Guducu  (2005,  p.19)  one  of  the  main  objectives  of  the  PBL  can  be 
described  as:  “Maximize  Performance:  The  contractor  can  deliver  the  required  service  by 
following  its  best  business  practices,  adjusting  them  accordingly.  The  contractors  can  be 
incentivized  to  use  their  best  performance.” 

It  is  so  clear  that  PBL  works  and  delivers  improved  perfonnance  in  various  ways 
(Fowler,  2009,  p.  7;  Kratz  and  Buckingham,  2010b,  p.  293).  According  to  Spring  (2010)  the 
efficiency  and  effectiveness  of  military  logistic  system  can  be  improved  by  PBL. 

Consequently  one  of  the  successes  of  PBL  is  improving  performance. 

4.  Partnership,  Strategic  Alliance 

In  forty  nine  out  of  eighty  one  documents,  partnership,  alliance  is  mentioned  regarding 

PBL  with  an  obligation.  “The  complex  systems  found  in  aerospace  and  defense  require  more 

sophisticated  relationships  between  service  buyers  and  suppliers.”  (Cohen  and  Netessine,  2007, 

p.2)  DoD’s  aim  is  to  get  improved  perfonnance  within  a  fair  cost  through  PBL. 

“Today,  the  DoD’s  goals  are  not  only  to  gain  the  most  efficient  and  effective 
perfonnance  of  systems  throughout  their  entire  life  cycles,  but  also  to  build  partnerships 
and  relationships  that  align  the  goals  of  all  involved  organizations  for  the  duration  of  the 
programs.”  (Gardner,  2008,  p.9) 

PBL  environment  is  enclosed  by  long  term,  strategic  partnership.  Without  it  PBL  cannot  be  even 
described. 

“Companies  that  are  considering  PBLs  should  realize  that  they  are  entering  into  de  facto 
partnerships  with  their  logistics  services  providers  and  that  these  partnerships  cannot  be 
created  overnight.”  (Vitasek  and  Geary,  2007,  p.l) 

Partnership,  strategic  alliance  is  not  just  a  business  relationship  in  PBL.  Traditional 
approach  can  be  called  as  arm’s  length  relationship.  According  to  Ott  (2008);  it  is  a  new  level, 
partners  are  developing  better  relationships. 
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“Contractual  relationships  that  are  largely  transactional,  involving  very  little  integration 
of  operations  between  DoD  and  smaller  support  providers,  are  generally  not  considered 
to  be  perfonnance-based  contracts  and  tend  to  fit  the  category  of  “arm’s  length 
relationships”  as  described  by  Lambert  et  al.  (1996).”  (Gardner,  2008,  p.10) 

Many  relationships  are  moving  towards  long  tenn  partnership  from  traditional  ann’s 

length  approach.  According  to  Nicosia  and  Moore  (2006);  this  is  a  development  to  improve 

relations  through  strategic  alliances.  They  also  stated  that  long  term  strategic  partnership  is  a  key 

for  supply  chain  management. 

“In  the  PBL  environment,  a  Government/industry  team  is  a  key  long-tenn  relationship 
that  is  developed  among  public  and  private  stakeholders  contractually  and /  or  with  performance 
agreements.”  (Performance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide, 
2005,  p.1-1) 

The  definition  of  PBL  in  DoDD  5000.01  (2003,  p.7)  includes  partnership.  “Sustainment 
strategies  shall  include  the  best  use  of  public  and  private  sector  capabilities  through 
govemment/industry  partnering  initiatives,  in  accordance  with  statutory  requirements.”  To  have 
a  successful  PBL,  partnership  is  an  obligation.  Best  practices  include  “Government-Industry 
partnerships,  consistent  with  contract  documents.”  (DoDD  5000.02,  2002,  p.31)  These 
definitions  are  cited  by  Berkowitz,  Gupta,  Simpson  and  McWilliams  (2004).  Dibenedetto  (2007) 
describes  this  concept  as  an  effective,  collaborative  partnership.  Heron  (2010)  believes  that  this 
partnership  is  a  tie  for  PBL,  a  part  of  PBL.  Therefore  it  should  be  mandated  in  PBL. 

Bozkurt  and  Guducu  (2005)  think  that  forming  a  long  tenn  relationship  is  the  intent  of 
PBL.  PBL  may  “increase  weapon  system  readiness  through  cost-effective,  integrated,  logistics 
chains  and  public/private  partnerships.”  (Gansler,  et  al.,  2011,  p.6) 

According  to  Devries  (2004)  one  of  the  seven  enablers  of  PBL  is  strategic 
alliances/partnerships.  Buyukgural  (2009),  Bozkurt  and  Guducu  (2005),  Gardner  (2008)  and 
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Vitasek  et  al.  (2007)  cites  Devries  (2004)  research  results  in  their  studies  to  explain  the  key 
drivers  of  PBL.  Vitasek  et  al.  (2006)  declare  public-private  partnerships  as  a  key  component  of 
successful  PBL.  “A  contracting  strategy  developed  in  partnership  with  the  contractor  ...”  is  one 
of  the  success  factors  according  to  Kate  Vitasek  (Blumberg,  2007b,  p.5). 

Slinkard  and  Poleskey  (2006)  accept  that  award-tenn  contracting  is  a  good  way  to 
incentivize  long  tenn  relationships.  According  to  them  leading  partnerships  is  the  win-win 
solution.  The  full  promise  of  a  carefully  structured  PBL  can  only  be  achieved  with  partners 
integrated  into  a  final  solution  set.  There  should  be  a  mutual  understanding  of  corresponding 
goals  (Trovato,  2004,  p.23).  Burkett  (2008,  p.l)  states  that  “Buyers  don’t  just  want  a  product  or  a 
service  but  a  relationship  with  a  partner  that  provides  a  total  value  solution.” 

PBL  contracts  are  unique  partnerships  to  meet  the  goal.  “Contractors  often  enter  into 
unique  partnerships  with  DOD  facilities  to  achieve  readiness  and  cost  goals.”  (Blumberg,  2006b, 
p.58)  Therefore  best  service,  practice,  utility  are  experienced  (Blumberg,  2007a,  p.42).  Miller 
(2008)  sees  PBL  as  an  alternative  strategy  to  have  best  commercial  practices,  skills  in  order  to 
have  effective,  efficient  support. 

It  is  important  to  be  aware  of  strategic  partnership  relations.  Education  improves  the  will 
to  build  these  relationships.  Maylett  and  Vitasek  (2008)  believe  that,  sessions  regarding  these 
kinds  of  relationships  will  improve  strategic  partners’  knowledge  and  abilities. 

According  to  Fowler  (2009,  p.5)  DoD’s  PBL  framework  includes  “public-  and  private- 
sector  provider  strategies,  with  partnering  being  an  integral  component  of  PBL  approaches”.  But 
even  more,  “the  next-generation  PBL  strategies  need  to  offer  improved  attention  to  the  enterprise 
integration  effects.”  Kratz  and  Buckingham  (2010b)  believe  that  all  future  acquisition  strategies 
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depend  on  successful  partnerships.  Spring  (2010)  suggests  encouraging  and  reinforcing  these 
partnerships  to  improve  PBL  and  those  other  future  strategies. 

In  Newsome’s  (2008,  p.  12-14)  study,  survey  participants  make  comments  on  partnership, 
relationship.  Some  of  them  are  below: 

“General  industry  trend  however  we  are  moving  to  alliance  contracting  with  an 
increasing  number  of  our  customers.  . . .  We  are  Public  Partners  in  Several  Public/Private 
PBL  agreements.  . . .  Almost  six  years  of  good  experience.  Predictability  of  costs.  True 
partnership  with  supplier.” 

Thirteen  documents  give  following  examples  to  suppress  the  importance  of  partnership. 
Table  5  shows  the  examples  used  by  corresponding  documents. 


Table  5.  Examples  used  in  documents-  Partnership/Strategic  Alliance. 


Berkowitz  et  al.  (2003) 

FI-60  FUR 

X 

C-17 

X 

JSTARS 

X 

F-22 

F/A-18 

X 

APU 

A/C  Tires 

Shadow 

C-130J 

GE  404 

JAVELIN 

Blumberg  (2006b) 

X 

Blumberg  (2007b) 

X 

X 

Bozkurt  and  Guducu  (2005) 

X 

X 

X 

X 

Goure  (2009) 

X 

X 

Kratz  and  Buckingham  (2010b) 

X 

X 

Mahadevia,  Engel  and  Fowler 
(2006) 

X 

Marietta  (2008) 

X 

Ott  (2008) 

X 

Slinkard  and  Poleskey  (2006) 

X 

Spring  (2010) 

X 

Vitasek  and  Geary  (2008) 

X 

Vitasek,  Cothran  and  Rutner 
(2007) 

X 

Total 

4 

4 

3 

3 

2 

1 

1 

1 

1 

1 

1 

H-60:  Sikorsky  Aircraft  Corporation  is  responsible  as  PSI  in  this  PBL  program. 

“Sikorsky  Aircraft  Corporation  (SAC)  is  working  side  by-side  with  Corpus  Christi  Army 
Depot  (CCAD)  to  reduce  repair/overhaul  turnaround  time  for  the  H-60.  This  joint 
collaboration  has  improved  business  processes,  depot  repair  methodology,  and  more 
responsive  product  support,  with  only  four  contractor  jobs  directly  attributable  to  the 
partnership.”  (Berkowitz  et  al.,  2003,  p.  1 7;  Bozkurt  and  Guducu,  2005,  p.60) 

MHSCo  is  the  joint  venture  company  of  Lockheed  Martin/Sikorsky  which  has  partnered 

with  Navy  for  this  PBL  program.  This  is  a  unique  partnership  according  to  Blumberg  (2006b).  In 
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addition  to  that  for  FLIR  systems  of  H-60,  Raytheon  is  responsible  “for  supply  chain 
management,  configuration  management,  and  total  system  performance.”  (Vitasek  and  Geary, 

2008,  p.63) 

C-17  Globemaster:  This  PBL  is  an  agreement  of  Boeing- Air  Force  partnership.  “Boeing 
has  direct-sales  partnership  agreements  with  each  of  the  three  Air  Logistics  Centers.”  (Goure, 

2009,  p.l)  “They  have  joint  weekly,  monthly,  block,  etc.  meetings  and  reviews.  Every  employee 
who  works  on  the  C-17  wears  a  plastic  card  the  size  of  their  badge,  imprinted  with  partnership 
agreement  signed  by  Boeing  and  Air  Force  leaders.”  (Berkowitz  et  ah,  2003,  p.  17;  Bozkurt  and 
Guducu,  2005,  p.60)  They  used  these  badges  to  implement  the  idea  of  alliance. 

E-8C  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS):  According  to 
Berkowitz  et  al.  (2003),  Bozkurt  and  Guducu  (2005);  there  are  multiple  agreements  in  place  for 
Total  System  Support  Responsibility  partnership  in  this  PBL  program.  Slinkard  and  Poleskey 
(2006)  takes  JSTARS  PBL  program  as  a  good  example  of  long  tenn  relationship  in  supply  chain 
management. 

F-22  Raptor:  Lockheed  Martin  sees  this  PBL  program  as  a  strong  partnership.  “The 
government-contractor  team  is  a  strong  partnership  providing  rapid  maturation  of  the  F-22 
support  system  and  exceptional  support  to  the  warfighter;  said  Dennis  Haines,  Lockheed  Martin 
vice  president  for  F-22  Sustainment.”  (Marietta,  2008,  p.  1)  Goure  (2009)  pointed  out  that  this 
program  won  the  DoD  2008  PBL  System  Level  Award  and  achieved  various  performance 
improvements  by  public-private  teams  and  partnership. 

F/A-18E/F  Integrated  Readiness  Support  Teaming  (FIRST)  Program:  According  to 
Berkowitz  et  al.  (2003  p.17),  Bozkurt  and  Guducu  (2005,  p.60);  “fifty-seven  government  jobs 
were  created  or  sustained  by  this  partnership.” 
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Auxiliary  Power  Unit  (APU):  Vitasek,  Cothran  and  Rutner  (2007)  believe  that  the 
success  in  APU  PBL  program  has  been  provided  by  Navy  and  Honeywell  partnership.  Not  only 
the  improvements  but  also  the  benefits  stated  in  their  study  are  the  products  of  this  partnership. 

GAO  Report  of  Depot  Maintenance  (2008)  states  that  there  are  various  benefits  of 
building  closer  relationships.  These  benefits  are  explained  in  details  for  both  studies. 

Aircraft  Tires:  Michelin,  Lockheed  Martin,  and  EGL  are  integrated  to  support  this  PBL 
program.  “Michelin,  Lockheed  Martin,  and  EGL  are  integrated  through  a  Lifetime  Support 
Command  Center  (LSCC)  that  controls  all  requisitions  from  the  fleet  and  provides  warehouse 
management,  inventory  control,  and  data  to  Michelin  to  maintain  their  internal  systems  with 
program  data.”  (Mahadevia,  et  al.,  2006,  p.31) 

Shadow  Unmanned  Aerial  Vehicle  (UAV):  AAI  and  Anny  are  partners  in  this  PBL 
program.  The  partners  worked  even  at  war  zone  in  Iraq.  According  to  Blumberg  (2007b)  this  is 
the  most  unique  event  among  PBL  programs. 

C-130  J:  AAD  is  partner  with  Rolls-Royce  and  Lockheed  Martin  in  this  program.  Wayne 
Ayer,  Director  of  C-130J  Sustainment,  560th  Aircraft  Sustainment  Squadron,  states  that  “It's  a 
partnership,  where  we  rely  both  on  the  contractor  and  the  DOD,  so  we  calculate  his  contribution 
to  aircraft  availability  separately  from  the  DOD's  contribution.”  (Blumberg,  2007b,  p.6) 

Navy/GE  F404:  Navy  Capt.  Tim  Matthews,  commander  of  the  Fleet  Readiness  Center 
Southeast,  fonnerly  the  Jacksonville,  states  that  “I  talk  as  a  provider  of  touch  labor  facilities  and 
as  a  subcontractor  to  GE.  That’s  an  interesting  relationship  for  a  military  man  to  be  in.”  (Ott, 

2008,  p.60) 

JAVELIN:  Lockheed  Martin  and  Raytheon  are  partners  with  Anny  in  this  program. 
According  to  Spring  (2010);  although  some  of  the  depot  staff  sees  those  contractors  threat  to  the 
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security,  more  begins  to  think  as  contractors  are  one  of  them  or  they  are  one  of  the  employees  of 
the  contractor. 

Building  trust  is  mentioned  in  most  of  the  documents  which  include  partnership. 
According  to  Buyukgural  (2009,  p.39)  “Building  the  trust  and  partnerships  between  parties  is  an 
important  aspect  of  PBL”.  Gardner  (2008,  p.10)  references  Lambert  (1996)  regarding  trust  in 
partnerships  and  states  that;  “A  partnership  is  a  tailored  business  relationship  based  on  mutual 
trust,  openness,  shared  risk,  and  shared  rewards  that  yields  a  competitive  advantage,  resulting  in 
business  perfonnance  greater  than  would  be  achieved  by  the  firms  individually.”  According  to 
Canaday  (2006);  developing  trust  between  partners  should  be  the  first  step  for  making  the  PBL 
work. 

Lambert’s  classification  of  business  relationship  has  3  levels,  types.  Type  1  is  beginning 
from  slightly  more  than  arm’s  length  to  type  3  is  the  highest  level  of  partnership.  According  to 
(Gardner,  2008,  p.  12): 

“. . .  most  PBL  contracts  between  DoD  and  the  major  defense  contractors  seem  to  fit  into 
the  category  of  Type  II  partnerships,  defined  as  follows:  The  organizations  progress 
beyond  coordination  of  activities  to  integration  of  activities  involving  multiple  divisions 
and  functions  within  the  firm.  Although  not  expected  to  last  ‘forever,’  the  partnership  has 
a  long-tenn  horizon” 

As  stated  before;  partnership,  long  term  relationship  is  a  key  for  PBL  regardless  of  the 
level  or  stage.  It  is  even  in  level  1  as  in  APU  PBL  program.  As  Goure  (2010,  p.2)  states  “Enough 
evidence  exists  to  say  that  public-private  partnerships  are  a  good  way  to  integrate  the  organic  and 
private  defense  industrial  bases.”  PBL  is  used  for  readiness,  improving  performance,  availability. 
If  it  is  the  purpose,  the  only  way  to  ensure  it,  is  committing  to  a  long-tenn  relationship, 
partnership,  strategic  alliances  (Goure,  2010,  p.14).  This  is  the  only  and  best  way  to  use  both 
partners’  capabilities  to  achieve  the  best  results. 
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5.  Length  of  Contract 

In  forty  one  out  of  eighty  one  documents,  length  of  contract  is  mentioned  as  an  important 

factor  in  PBL.  “The  preferred  PBL  contracting  approach  is  the  use  of  long-term  contracts  with 

incentives  tied  to  performance.”  (Perfonnance  Based  Logistics:  A  Program  Manager’s  Product 

Support  Guide,  2005,  p.3-19;  Bozkurt  and  Guducu,  2005,  p.  1 5;  Blumberg,  2006a,  p.57)  This 

preference  is  so  clear  for  DoD  when  implementing  PBL  strategy.  To  improve  perfonnance,  to 

reduce  cost,  and  to  achieve  better  results,  contractors  should  invest  money.  The  intent  when 

investing  is  taking  it  back  as  profits  in  the  future.  “This  can  only  be  achieved  when  there  is 

sufficient  contract  length  to  assure  the  service  provider  of  an  adequate  return  on  investment  for 

these  actions.”  (Perfonnance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide, 

2005,  p.3-19)  Kobren  (2010,  p.187)  adds  “long-term  best  value  sustainment  strategies  that  bring 

a  balance  between  readiness  and  cost”  should  be  ensured  for  perfonnance  objectives.  DoD’s  plan 

is  having  a  long  term  support.  ‘Long  term  support’  is  listed  in  Kratz’s  (2001,  p.  14)  study  as  one 

of  the  product  support  planning  objectives  that  should  be  met  at  a  minimum. 

“PBL  arrangements  focus  on  the  purchase  of  measurable  performance  outcomes  (such  as 
the  availability  of  functioning  weapon  systems)  through  long-tenn  support  anangements 
rather  than  the  purchase  of  individual  elements  of  support — such  as  parts  repairs,  and 
engineering  support”  (Gansler,  et  ah,  2011,  p.6;  GAO  Report:  Performance-Based 
Logistics,  2008,  p.l;  Goure,  2010,  p.l;  Mirazahosseinian  and  Piplani,  2011,  p.256) 

The  contracting,  planning,  upgrading  processes,  themselves  take  much  time.  According 

to  Hedden  (2005);  improving  reliability  depends  on  upgrades  (including  long  tenn  planning) 

which  consume  a  long  period  of  time. 

Garnder  (2008)  states  that  the  most  incentive  factor  is  contract  length  for  balancing  risk 
and  up-front  investment  in  PBL.  Developing  long  term  relationships  is  only  possible  with  longer 
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contracts.  Kim  et  al.  (2006)  think  that  in  the  long  run  contractors  may  show  interest  to  invest  for 
more  reliable,  improved  products,  service. 

“...  the  PBL  goal  of  developing  long-term  partnerships  that  encourage  investment  from 
commercial  partners  is  best  achieved  through  lengthy,  guaranteed  contracts  ...  At  various 
recent  PBL  conferences  and  seminars,  private  industry  representatives  have  consistently 
expressed  that  contracts  of  shorter  term  lengths  limit  defense  contractors’  ability  to 
realize  worthwhile  return  on  investment”  (Gamder,  2008,  p.iv) 

According  to  Kobren  (2009)  long-term  contracts  allow  contractors  to  balance  risk  against  their 

investments.  “A  perfonnance-based  contract's  length  is  tied  to  the  ‘payback  period’  for  the 

service  provider  to  recoup  its  investments.”  (Dibenedetto,  2007,  p.38) 

Gardner’s  (2008,  p.42)  study  shows  that: 

“A  consistently  high  level  of  satisfaction  with  contract  length  was  found  amongst 
programs  that  had  contracts  with  a  5-year  base,  followed  by  option  years  or  award  terms. 
Respondents  in  these  cases  expressed  that  the  contract  length  allowed  for  an  appropriate 
amount  of  risk  sharing  and  return  on  investment.” 

Cebeci  (2009,  p.22)  cites  a  RAND  study  and  states  the  advantages  of  multi-year 
contracting: 

“. . .  the  avoidance  of  nonrecurring  costs  associated  with  negotiating  and  implementing  a 
new  contract  each  year,  long-tenn  hiring  and  personnel  planning  can  be  made  more 
efficient,  and  the  contractor  can  plan  the  purchase  of  long  lead  items  and  materials  longer 
production  runs.” 

While  Cebeci  (2009)  is  listing  the  benefits  regarding  costs,  DoD  PBL  guide  and  Vitasek 
et  al.  (2006)  state  that  long  term  contracts  provide  cash-flow  continuity  and  “time  horizon 
sufficient  to  allow  contractor  investment  to  improve  products  and  processes,  confident  that  they 
will  receive  an  adequate  Return  on  Investment.”  (Perfonnance  Based  Logistics:  A  Program 
Manager’s  Product  Support  Guide,  2005,  p.3-34;  Sols  and  Verma,  2007,  p.  44;  Vitasek,  et  al., 
2007,  p.7)  Vitasek  et  al.  (2007)  see  PBL,  regarding  long  term  contracting,  as  an  opportunity  to 
plan  future  workloads,  requirements  and  manage  obsolescence  better.  These  benefits  incentivize 
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contractors  to  win  longer  contracts  rather  than  short  ones.  Gansler  et  al.  (2011,  p.12)  count  long 
term  contract  as  a  benefit  by  itself  among  the  partners. 

Berkowitz  et  al.  (2004)  define  PBL  in  their  study  and  emphasize  that  contractual 
mechanisms  should  have  long  term  relationship.  Regarding  the  definitions  used  by  DoD  and  its 
branches;  Pettingill,  et  al.  (2004,  p.7)  criticize  Army  and  Air  Force  not  using  the  ‘long  tenn 
perfonnance  based  agreement’  in  PBL  definition;  while  Navy  definition  “specifically  mentions 
the  importance  of  long-term  arrangements”.  In  their  study,  AFLMA’s  proposed  definition  has 
adequate  emphasis  on  long  term  contracting. 

According  to  Lessons  Learned  (2004)  Navy  suggests  long  tenn  partnering  in  their 
business  case  analysis,  especially  to  get  early  commitment  and  investment.  The  use  of  long-term 
contracts  is  listed  among  the  reasons  Navy  cites  for  its  success  with  PBL  (Goure,  2010,  p.4). 
Recent  studies  show  that  DoD  is  relying  more  and  more  on  long  tenn  strategic  relationships  day 
by  day.  Also  GAO  reports  make  the  same  conclusion  (GAO  Report:  Performance-Based 
Logistics,  2008). 

Successful  PBL  programs  are  the  ones  with  longer  contracts.  Vitasek  states  that  one  of 
the  common  factors  for  success  in  PBL  is  long  term  contracts  (Blumberg,  2007a,  p.44).  Contract 
length  should  be  examined  carefully  because  it  is  a  key  for  success.  Contract  length  is  listed 
under  PBL  Contract  Structure  Elements  in  Vitasek’s  lecture  and  thought  by  her  as  a  tenet  in 
PBL’s  success  (Wuchenich,  2008,  p.3).  Owings  (2010)  supports,  too;  to  make  PBL  work 
properly  long  term  commitment  is  required.  “The  government  also  sees  ‘long-term  support 
arrangements’  as  a  key  element  of  PBL  success.”  (Gardner,  2008,  p.2)  It  is  a  key  to  success. 

All  acquisitions  are  not  fit  in  PBL  such  as  short  contracts.  “The  contract  may  expire 
without  any  benefit  for  the  customer  because  of  the  short  term,  or  the  supplier  may  not  achieve  a 
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certain  performance  level  because  they  have  insufficient  time  to  apply  their  skills.”  (Harada, 
2010,  p.25) 

Both  organic  and  commercial  representatives  agree  that  5  years  for  the  length  of  contract 
is  the  base  for  success  in  PBL.  Literature  states  ‘at  least  5  years  is  required’  (Harada,  2010,  p.31; 
Vitasek  and  Geary,  2008,  p.64;  Gardner,  2008,  p.46;  Bozkurt  and  Guducu,  2005,  p.  1 5).  Some 
authors  state  three  to  live  years  (Gansler,  et  ah,  2011,  p.61).  But  it  should  not  be  less  than  three 
(Blumberg,  2007a,  p.44).  It  is  common  that  return  on  investments  gets  better  after  5  or  more 
years  (Gansler,  et  ah,  2011,  p.61).  Gardner  (2008)  states  that  long  term  contract  are  not  always 
the  answer  but  it  usually  is.  Long  term  contracts  are  not  the  only  magic  that  makes  PBL  works 
by  itself. 

GAO  reports,  mentioned  earlier,  take  U.K.  and  its  PBL  experience  in  their  studies  and 
states  more  than  10  year  long-tenn  contracting  is  crucial.  But  also  they  cite  military  literature 
which  suggests  5  years.  There  are  some  arguments  regarding  limitation  of  competitiveness, 
flexibility  by  long  contracts  in  the  literature.  Kratz  and  Buckingham  (2010a)  are  one  of  them. 
They  think  that  long  contracts  limit  government’s  flexibility  to  adjust  the  real  world  changes  and 
budgeting.  Kratz  and  Buckingham  (2010b)  offer  reducing  contract  length  to  enable  continuous 
competition.  Although  they  argue  about  so  long  contracts,  they  admit  that  5  years  is  good. 

Seventeen  documents  give  examples  regarding  partnerships  as  a  factor  with  examples.  Table 
6  shows  the  examples  used  by  corresponding  documents. 

F-117  Nighthawk:  This  is  a  system  level  PBL  contract  for  5  year  with  3  year  option 
(Vitasek  et  ah,  2006).  Funding  is  stabilized  by  the  government  (Pettingill  et  al,  2004).  This  leads 
long  term  commitment.  As  stated  before,  this  program  was  awarded  at  system  level  in  2005 
(Vitasek  et  al.,  2006).  This  program  resulted  many  perfonnance  improvements  and  cost  savings, 
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although  it  was  one  of  the  legacy  programs  that  turned  to  a  PBL  (Blumberg,  2006a).  All  authors 
think  this  program  had  a  success  with  its  contract  length. 


Table  6.  Examples  used  in  documents-  Length  of  contract. 


F-117 

APU 

JSTARS 

GE 

404 

UK 

TORNADO 

HIMARS 

ITAS 

RENFE 

TRAIN 

F/A-18 

C-17 

UK 

NIMROD 

JSF 

Shadow 

UAV 

H-60 

FUR 

UK 

Sea  King 

UK 

Survey  Ship 

UK 

Merlin 

Blumberg  (2006a) 

X 

Blumberg  (2006b) 

X 

X 

Blumberg  (2007a) 

X 

Canaday  (2006) 

X 

X 

Gansler,  Lucyshyn, 
Harrington  and  Corl 
(2011) 

X 

GAO  Report:  PBL (2008) 

X 

X 

X 

X 

Gardner  (2008) 

X 

Geary,  Koster,  Randall 
and  Haynie  (2010) 

X 

Lessons  Learned  (2004) 

X 

DoD  PBL  Guide  (2005) 

X 

Pettingill,  Knipper, 
Gaudette  and  Gayer 
(2004) 

X 

X 

X 

Slinkard  and  Poleskey 
(2006) 

X 

Sols  and  Verma  (2007) 

X 

Vitasek  and  Geary  (2008) 

X 

Vitasek,  Cothran  and 
Rutner  (2006) 

X 

X 

Vitasek,  Cothran  and 
Rutner (2007) 

X 

Vitasek,  Geary,  Cothran 
and  Rutner  (2006) 

X 

X 

X 

Total 

5 

3 

3 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

Auxiliary  Power  Unit  (APU):  This  PBL  contract  is  a  10  year  agreement;  5  year  base 

with  5  -  1  year  renewal  options  (Vitasek  et  al.,  2007).  It  achieved  several  success  stories  and  it 

was  awarded  in  2005  at  component  level  (Vitasek  et  al.,  2006b).  Gansler  et  al.  (2011,  p.28)  give 

this  example  under  the  title  of  “A  10  Year  PBL  Success  Story”. 

E-8C  Joint  Surveillance  Target  Attack  Radar  System  (JSTARS):  The  contract  is 

signed  between  Northrop-Grumman  (NG)  and  Air  Lorce  (AL). 

“The  basic  contract  includes  6  years  with  the  potential  for  22  additional  years  as  part  of 
the  award-term  arrangement.  A  good  working  relationship  exists  among  the  AL,  NG,  and 
the  warfighter.  The  long-term  nature  of  this  contract  was  identified  as  a  best  practice  as  it 
motivates  an  atmosphere  of  cooperation  and  long-term  investment  between  NG  and  the 
AL.”  (Pettingill,  et  al,  2004,  p.19) 
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According  to  Geary,  Koster,  Randall  and  Haynie  (2010)  this  PBL  program  is  a  long-tenn  success 
of  utilizing  long  term  hybrid  structure  of  partners.  In  their  study,  Slinkard  and  Poleskey  (2006) 
examined  JSTARS  example  regarding  award  tenn,  its  advantages  and  disadvantages. 

Navy/GE  F404:  This  is  one  of  the  best  major  sub-system  PBL  examples  according  to 
Vitasek  et  al.  (2006).  The  contract  is  4 A  year  base  with  5  additional  1  year  option.  Improvements 
and  savings  are  mentioned  at  previous  sections.  It  was  also  awarded  at  its  stage  and  all  optional 
years  were  awarded  to  the  first  contractor  (Vitasek  et  al.,  2006b). 

Tornado  Support  in  United  Kingdom  (U.K.):  At  first  this  was  a  pilot  PBL  program  for 
U.K.  but  now  they  decided  to  move  on  a  10  year  contract  due  to  the  achievements  (Canaday, 

2006,  p.2). 

“According  to  officials,  the  long-tenn  contracts  for  Tornado  aircraft  and  helicopter  fleets 
reduced  costs  because  the  contractors  were  able  to  stabilize  their  supply  chain  and  obtain 
better  prices  from  the  supplier  base.  The  Ministry  of  Defence  also  found  that  industry 
preferred  long-tenn  contracts.  In  a  discussion  of  contracting  for  availability,  the  “Defence 
Industrial  Strategy,”  a  white  paper  dated  December  2005,  stated  that  companies  are 
generally  interested  in  using  availability  contracts  because  it  provides  the  commercial 
finns  with  greater  returns  over  a  longer  period.”  (GAO  Report:  Improved  Analysis  and 
Cost  Data  Needed  to  Evaluate  the  Cost  Effectiveness  of  Perfonnance-Based  Logistics, 
2008,  p.49-50) 

Other  examples  and  key  points  are  summarized  in  table  7. 

These  are  the  best  in  their  stage  or  good  examples  of  working  PBL  and  as  seen  they  have 
at  least  3  years  contract  with  award  terms  options.  Consequently;  contract  length  is  an  important 
factor  and  it  can  be  stated  that  less  than  a  3  years  contract  may  not  work  for  PBL. 


6.  Flexibility 

In  thirty  nine  out  of  eighty  one  documents,  flexibility  is  mentioned  as  an  obligation  for 
PBL.  PBL  is  an  acquisition  system  where  decentralization,  innovation,  creation  are  used  in  an 
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optimal  way.  To  achieve  its  promises,  PBL  environment  should  “enable  greater  flexibility  in 
capitalizing  on  commercial  technologies”  (DoDD  5000.01,  2003,  p.7;  Kratz,  2001,  p.14) 


Table  7.  Summary  of  examples  for  Length  of  Contract. 


Contract 

Length 

Key  Points 

HI  MARS 

3yrs 

High  level  of  satisfaction  with  a  short  contract. 

HAS 

5yrs 

Well  structured  PBL  performance  agreement.  Several  improvements. 

RENFETRAIN 

14yrs 

Availability  and  mission  reliability  improvements. 

F/A-18 

3  yrs  + 
up  to  30  yrs 

Focus  on  creating  a  longterm  business  arrangement  that  will  stand  the 

test  of  time. 

C-17 

5  yrs  + 

3x  lyropt 

Boeing's  efforts  were  considered  a  "PSI"  best  practice. 

UK  NIMROD 

6  yrs 

Incrased  aircraft  availability,  decreased  costs. 

JSF 

Since  2001 

JSF  program  is  on  target  with  sufficient  long-range,  up-front  planning  to 
provide  a  high  probability  of  success. 

Shadow  UAV 

5  yrs  + 

7  x  lyr  opt 

Shadow  program  meets  the  criteria  for  a  PBL  program  that  should 
ultimately  prove  to  be  very  successful. 

H-60FLIR 

10  yrs 

Many  imrovements  and  cost  reductions. 

UK  Sea  King 

10  yrs 

Long-term  nature  of  availability  contracts  a  key  factor  in  reducing  costs 

and  that  annual  contracts  cannot  achieve  the  same  benefits  as  the 

longer-term  contracts  do.  Commercial  firms  prefer  so. 

UK  Survey  Ship 

23  yrs 

UK  Merlin 

25  yrs 

PBL  is  an  acquisition  method  which  pays  for  outcomes,  not  how  it  is  delivered.  A  PBL, 
which  has  clearly  defined  outcomes,  performance  measures  and  evaluation  method,  allows 
partners  “maximum  flexibility  to  attain  the  greatest  degree  of  innovation  and  creativity.” 
(Memorandum:  Performance-Based  Services  Acquisition,  2000,  p.2)  Otherwise  partners  are 
limited  with  boundaries,  ‘how  to  do’s.  There  is  one  way  to  achieve  goals,  ultimate  purposes  and 
it  is  mostly  depend  on  flexibility. 

“In  a  true  PBL  contract,  the  customer  specifies  the  outcome  they  are  expecting  but  not  the 
method  used  to  achieve  that  outcome,  thus  leaving  it  open  to  the  contractor  to  implement 
commercial  best  practices  to  not  only  achieve  the  expected  results  but  make  significant 
improvements  as  well.”  (Newsome,  2008,  p.2) 
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Also  FAR  regulates  not  using  ‘how  to  do’s.  According  to  FAR  one  of  the  four 
requirements  for  performance  based  contracting  is  telling  the  contractor  what  is  needed  rather 
than  how  to  provide  the  service  (Baldwin,  Ausink  and  Nicosia,  2005).  Gansler,  et  al.  (2011)  cite 
FAR  regulations  when  he  is  describing  the  need  of  flexibility.  According  to  Kim,  et.al.  (2006, 
p.3)  there  is  a  clear  separation  between  the  desired  outcomes  and  how  to  achieve  those.  “The 
contract  explicitly  identifies  what  is  required,  but  the  contractor  determines  how  to  fulfill  the 
requirement.” 

Major  Gen.  Daniel  Mongeon  states  that  Defense  Logistics  Agency  (DLA)  “buys 

perfonnance  as  a  package,  which  includes  surge  flexibility.”  (Anonymous,  2005,  p.33) 

According  to  him  “DLA  is  stepping  up  to  the  plate  to  provide  flexibility  and  agility  to  support 

the  warfighter  today.”  (Anonymous,  2005,  p.34) 

“The  guiding  principle  in  Performance  Based  Service  Acquisition  is  that  when  an  outside 
vendor  exists  that  can  perform  a  service  more  effectively  than  a  government  user  could 
organically  (i.e.,  in-house),  the  government  client  should  specify  measurable  outcomes  to 
a  service  vendor,  and  allow  the  vendor  to  best  determine  the  appropriate  processes  (the 
"how")  of  delivering  the  service.”  (Doerr,  Lewis  and  Eaton,  2005,  p.  2) 

In  PBL  buyer  is  trying  to  use  contractor’s  knowledge,  experience,  capabilities;  trying  to 

have  best  commercial  practices.  There  is  a  necessity  of  freedom  for  contractor’s  actions  to  do 

what  he  does  best.  That’s  why  most  goes  PBL  from  traditional  approach.  Traditional  approach 

includes  processes,  specifications  about  what  should  be  done.  In  PBL,  just  desired  outcomes  are 

specified.  This  is  “a  transition  from  telling  the  contractors  what  to  do  and  how  to  do  it,  to  telling 

the  contractor  what  to  achieve,  and  then  relying  on  their  knowledge  and  experience  to  do  it.” 

(Sols  and  Venna,  2007,  p.40;  Venema,  2007,  p.l;  GAO  Report:  Improved  Analysis  and  Cost 

Data  Needed  to  Evaluate  the  Cost  Effectiveness  of  Performance-Based  Logistics,  2008,  p.7)  This 

is  flexibility  and  this  will  provide  more  space  for  creativity,  innovation.  As  a  result  the 
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“contractors  are  given  the  necessary  freedom  and  responsibility  to  innovate  and  apply  their 
domain  knowledge  and  experience,  within  a  context  of  financial  motivation  and  incentives.” 
(Sols  and  Verma,  2007,  p.49)  According  to  Trovato  (2004);  PBL  is  an  effort  to  shorten  logistics 
tail  of  military  by  having  contractor  to  achieve  the  goals  through  a  flexible  environment.  “The 
requirement  that  performance  goals  be  met  is  imposed  on  the  supplier  but  is  of  sufficient 
duration  and  flexibility  to  pennit  the  supplier  to  adapt  his  enterprise  to  meeting  the  goals.” 
(Spring,  2010,  p.2) 

General  George  S.  Patton  said  that  “Never  tell  people  how  to  do  things.  Tell  them  what 
needs  to  be  done,  and  they  will  surprise  you  with  their  ingenuity.”  (Vitasek  and  Geary,  2007, 
p.3)  Let  them  do,  what  they  are  capable  of. 

PBL  does  not  have  rules  for  contractor  how  to  achieve  the  results.  It  does  not  dictate  how 
to  accomplish  the  desired  outcomes.  Traditional  approach  has  dictation  what  to  produce.  On  the 
other  hand  PBL  just  states  what  the  desired  outcomes  are  (Vitasek  and  Geary,  2008).  PBL  “does 
not  dictate  the  methods  or  limit  the  contractor’s  ability  to  detennine  how  to  do  so.”  (Owings, 
2010,  p.15)  Therefore  PSI  has  “significant  latitude  to  exercise  a  creative  and  entrepreneurial 
approach  to  not  only  meet,  but  often  exceed”  these  buyer’s  requirements  (Cothran,  n.d.,  p.8).  As 
long  as  the  buyer  gets  what  he  wants,  PSI  has  flexibility  how  the  support  is  provided.  According 
to  Vitasek  et  al.  (2006a,  p.2;  2006b,  p.7)  this  is  an  opportunity  for  PSI  to  decide  how  to  provide 
the  necessary  support.  And  also  for  the  government  “it  is  a  chance  to  it  is  a  chance  to  obtain 
guaranteed  availability  improvements  while  decreasing  costs  and  logistics  footprint  through 
partnering  with  private  business  for  complete  program  support.” 

Blumberg  (2006a,  p.57)  states  that  buyer  is  not  focusing  on  details  such  as  “the  number 
of  contractor  personnel  assigned  to  a  facility,  or  where  that  facility  is  located”  but  he  is  “paying 
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for  the  results  produced,  and  the  risk  burden  shifts  to  the  contractor  to  deliver.”  According  to 
Mirazahosseinian  and  Piplani  (2011),  customer  just  sets  the  objectives,  it  is  expected  from 
supplier  to  achieve  them  with  flexibility  on  how  to.  The  model  they  offer  has  flexibility,  as  a 
factor  in  it.  “The  contractor  has  flexibility  on  the  details  to  hit  the  numbers.”  (Blumberg,  2007a, 
P-44) 


In  addition  to  that  for  more  flexibility,  responsibility,  authorization  should  be 

decentralized.  According  to  DoDD  5000.01  (2003,  p.3)  responsibility  for  the  acquisition  of 

systems  shall  be  decentralized  to  the  maximum  extent  practicable.  In  PBL  environments  PSI  has 

so  much  responsibility,  therefore  it  should  be  empowered. 

“In  a  PBL  strategy,  DoD  is  empowering  the  PSI  with  the  ultimate  responsibility  - 
producing  warfighter  operational  effectiveness  by  ensuring  a  continuously  available, 
reliable,  and  effective  system.  The  concepts  of  PBL  -  buying  perfonnance  outcomes, 
incentivizing  the  PSI,  specifying  ‘what’,  not  ‘how’  those  outcomes  are  achieved,  all 
facilitate  the  tremendous  success  evident  in  PBL  support  strategies  to  date.”  (Cothran, 
n.d.,  p.  20) 

But  responsibilities  should  be  defined  clearly.  “The  transfer  of  responsibilities  is  not  risk-free, 

and  the  contractual  framework  has  to  be  very  well  defined.”  (Sols  and  Verma,  2007,  p.40) 

Authorization  should  be  given  to  the  provider  for  “flexibility  to  develop  innovative  solutions  in 

order  to  achieve  the  client’s  desired  outcomes.”  (Vitasek  and  Geary,  2007,  p.6) 

Berkowitz  et  al.  (2004,  p.259)  like  Navy’s  definition  because  of  using  ‘empowered  PSI’ 

term,  granting  additional  power  to  decision  maker: 

“A  PBL  strategy  is  an  agreement,  usually  long  term,  in  which  the  provider  (organic, 
commercial,  and/or  public/private  partnership)  is  incentivized  and  empowered  to  meet 
overarching  customer  oriented  performance  requirements  (reliability,  availability,  etc.)  in 
order  to  improve  product  support  effectiveness  while  reducing  TOC.” 
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Devries  (2004)  defines  working  capital  fund,  various  appropriations,  transfer  and 

expiring  funds  rules,  limited  Program  Manager’s  control  over  Operation  and  Maintenance,  etc. 

as  funding  restrictions/inflexibility  by  means  a  barrier  for  PBL. 

“PBL  transfers  many  of  the  ‘make  or  buy’  decisions  to  the  prime  support  contractor,  yet 
DoD  financial  rules  still  require  government  managers  to  include  separate  appropriation 
funding  requirements  in  contracts  -  in  other  words  forcing  them  to  ‘estimate’  what  kind 
of  support  decisions  the  support  contractor  will  make,  and  in  doing  so  setting  arbitrary 
boundaries  that  constrain  contractor  flexibility  to  make  best  value  decisions.”  (Vitasek  et 
ah,  2006a,  p.7) 

Vitasek  (2006a)  explains  the  situation,  which  Devries  (2004)  defines  as  a  barrier.  Vitasek 
(2006a,  p.8)  recommends  DoD  to  “press  ahead  with  vigor,  and  work  towards  alleviating  any 
remaining  financial,  statutory,  or  politic  barriers  that  limit  the  full  potential  of  PBL.”  PBL  will 
implement  best  commercial  capabilities  to  DoD’s  ability. 

Acquisition  always  carries  ambiguity,  risk  within.  Especially  for  new  systems,  PBL  do 
so.  Some  factors  may  decrease  their  effect.  According  to  Cebeci  (2009),  one  of  the  success 
boosters  that  will  help  to  reduce  their  effect  is  flexibility.  PBL  creates  such  an  environment. 
Cebeci  (2009,  p.54)  cites  (Garrett,  2005): 

“Performance-based  acquisition  creates  a  new  acquisition  environment  for  both  industry 
and  government  by  promoting  flexibility  and  innovation  and  by  creating  win-win 
solutions  through  effective  communication.  It  is  no  longer  optional;  it  is  a  business 
necessity  in  both  public  and  the  business  sectors.” 

Because  of  these  ambiguity  and  risk,  contractor  needs  some  freedom  for  his  actions  according  to 

the  determined  metrics.  “Contractor  needs  enough  flexibility  to  invest  in  areas  that  have  the  best 

returns,  and  to  adapt  his  processes  to  meet  the  objective  metrics.”  (Blumberg,  2007a,  p.44) 

Six  documents  used  different  examples  to  give  emphasis  on  flexibility.  The  examples  and 

key  points  used  in  corresponding  documents  are  listed  in  table  8. 
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Table  8.  Summary  of  examples  for  Flexibility. 


PBL  Program 

Reference 

Key  Points 

ITAS 

Performance  Based  Logistics:  A  Program  Manager's 
Product  Support  Guide  (2005) 

Maximum  flexibility 

F/A-18 

Lessons  Learned  (2004) 

Lots  of  rice  bowls  and  traditional  authorities  that  have 
to  be  gotten  past.. .fight  off  the  bureaucracy  that  wants 
to  dictate  "how” 

GE  404 

Vitasek,  Cothran  and  Rutner  (2006a) 

Afforded  the  flexibility  to  make  and  manage 
improvements  in  all  aspects  of  F404  support  from 
modifying  repair  processes  to  inserting  lean 
manufacturing  and  supply  chain  practices,  GE  can 
achieve  lots. 

NAVY  DEPOT 

(APU,  Tires,  etc.) 

Beilis  (2001) 

Supplier  acquires  ownership,  flexibility  to  improve 
supply  chain. 

C-130J 

Blumberg  (2007b) 

Comntractor  states:  "Now,  we  have  a  flexible 

incentivized  contract. 

C-17 

Barnes  and  Johnson  (2010) 

C-17  is  currently  maintained  under  a  strategy  of  flexible 
sustainment.  Success  in  emergencies  such  as 
experienced  at  Bagram  Air  Field  in  Afghanistan 
on  Jan.  30,  2006  for  a  crash  landed  C-17. 

The  PBL  programs,  which  include  flexibility,  have  several  successes,  achievements. 
Performance  requirements,  goals  to  be  achieved  should  be  described  in  details,  well;  on  the  other 
hand  how  to  achieve  those  should  be  left  to  the  provider.  Flexibility  is  one  of  the  15  attributes  of 
a  good  PBL  according  to  the  study  of  University  of  Tennessee  (Vitasek  and  Geary,  2008,  p.64). 
Geary  et  al.  (2010,  p.462)  count  flexibility  included  long  term  relationships  as  one  of  the  four 
fundamental  elements  of  performance-based  product  support. 

PBL  environment  gives  providers  flexibility  in  how  to  achieve  goals.  PBL  agreement 
“gives  contractors  the  flexibility  to  complete  required  tasks  in  the  manner  the  firm  deems  most 
appropriate.”  (Gansler,  et  al.,  2011,  p.57)  Acquisition  policies  should  include  flexibility  as  stated 
in  DoD  policies.  “DoD  policies  and  procedures  have  consistently  encouraged  flexibility  with 
respect  to  PSIs.”  (Fowler,  2009,  p.5) 

7.  Better  Service,  Best  Value  Creation,  Best  Practices 

In  thirty  eighth  out  of  eighty  one  documents,  it  is  stated  that  PBL  is  for  having  better 
service,  best  value  creation,  and/or  best  practices.  DoDD  5000.1  (2003,  p.7)  identifies  PBL  as  a 
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sustainment  strategy  which  shall  use  the  best  capabilities  of  public  and  private  sector.  DoD  PBL 
Guide  (2005),  Bozkurt  and  Guducu  (2005)  cite  this  definition  to  emphasize  gaining  the  best 
ability.  “The  transition  to  PBL  as  a  product  support  strategy  will  evolve  based  on  determination 
of  the  provider’s  product  support  capability  to  meet  set  performance  objectives.”  (Performance 
Based  Logistics:  A  Program  Manager’s  Product  Support  Guide,  2005,  p.  viii) 

According  to  Doerr  et  al.  (2005,  p.165)  the  guiding  principle  for  PBL  is;  if  there  is  an 
outside  vendor  performing  a  better,  more  effective  job  than  organic  one,  government  should 
detennine  the  desired  outcomes  and  let  the  vendor  does  what  he  does  better.  Although  PBL  is 
not  just  outsourcing,  the  underlying  idea  is  right;  let  PSI  does  the  job  to  gain  better  service.  The 
purpose  is  finding  out  the  best  decision,  best  value  provider.  “Best-value  providers  should  be 
selected  between  government,  industry  and  government-industry  partnerships.  These  best-value 
providers  would  be  the  companies  or  entities  whose  support  activities  would  yield,  for  the  money 
spent,  the  maximum  operational  effectiveness  of  the  system.”  (Sols  and  Verma,  2007,  p.40)  One 
of  the  objectives  that  the  sustainment  program  should  accomplish  is:  “Select  best-value,  long¬ 
term  product  support  providers  and  integrators  based  on  competition.”  (DoDD  5000.02,  2002, 
p.35;  Kratz,  2001,  p.  14) 

PBL  does  not  favor  either  PSI  candidates.  “The  decision  is  based  upon  a  best  value 
determination,  evidenced  through  a  PBL  BCA,  assessing  the  best  mix  of  public  and  private 
capabilities,  infrastructure,  skills  base,  past  performance,  and  proven  capabilities  to  meet  set 
performance  objectives.”  (Performance  Based  Logistics:  A  Program  Manager’s  Product  Support 
Guide,  2005,  p.  2-4) 

“The  detennination  of  a  PSI  comes  down  to  which  entity  has  the  best  ability  to  drive  life 
cycle  systems  engineering  influence  into  the  asset  (to  include  reliability  improvements), 
who  can  best  direct  supply  chain  management  decisions  to  assure  parts  availability  and 
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obsolescence  management,  and  who  can  be  incentivized  to  work  as  an  agent  for  the 
program  manager  to  meet  the  operational  sustainment  metrics”  (Fowler,  2009,  p.5) 

According  to  Newsome  (2008,  p.3);  PBL  is  changing  the  “approach  to  servicing  their 

customers  throughout  all  phases  of  a  product  life  cycle:  research  and  development;  production 

and  delivery;  operation  and  sustainment;  retirement  and  disposal”,  to  provide  customers  best 

value  and  maximize  life-cycle  performance.  One  of  the  main  objectives  of  PBL  is  maximizing 

perfonnance  via  letting  the  contractor  delivering  “the  required  service  by  following  its  best 

business  practices,  adjusting  them  accordingly”  (Bozkurt  and  Guducu,  2005,  p.  19)  PBL 

“partnerships  allow  each  side  to  do  what  they  do  best.”  (Goure,  2009,  p.l) 

DoD  PBL  Guide  (2005,  p.  3-6),  Bozkurt  and  Guducu  (2005,  p.l  1)  state  that;  when 

allocating  workload,  attention  should  be  paid  to  let  the  one,  who  does  that  job  better,  do  it. 

Evaluations  of  best  capabilities  should  be  considered  at  the  beginning  of  the  business.  PBL 

strategy  will  wary  along  this  work  allocation  done  according  to  organic  and  commercial 

capabilities  (Perfonnance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide,  2005, 

P-2-3). 

Sols  and  Verma  (2007)  think  that  buyer  relies  on  the  contractor’s  knowledge  and 
experience. 

Gardner  (2008,  p.67)  finds  that,  even  in  a  sole  source  environment  PBL  achieved  to  be 

awarded  due  to  the  experience,  knowledge,  ability  of  the  PSI  (mostly  OEMs).  According  to 

Berkowitz  et  al.  (2003,  p.8)  OEMs  are  the  best  in  managing  technology  and  obsolescence.  In 

addition  to  job  skills  and  knowledge,  management  issues  are  on  board.  In  PBL  PSI  is  responsible 

for  almost  everything.  According  to  Cothran  (n.d.)  PSI  has  a  proactive  role  in  this  partnership. 

“PSI  must  be  an  active  entity  -  conducting  detailed  and  comprehensive  analysis, 
planning,  and  forecasting  to  ensure  that  the  myriad  decisions  about  the  timing  and 
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delivery  of  specific  goods  and  services  across  the  range  of  support  providers  will 
hannonize  correctly  to  ensure  success.”  (Cothran,  n.d.,  p.12) 

Therefore  PSI  should  “possess  management/integration  and  program/project  management  skills 

and  experience.”  (Cothran,  n.d.,  p.12)  PSI  is  the  single  supplier  (an  umbrella)  that  provides  all 

with  best  practices  (Beilis,  2001,  p.18).  For  a  successful  PBSA  Gansler  et  al.  (2011,  p.61-62) 

recommend  to  “embrace  commercial  supply  chain  management  best  practices  more  fully.  ...  By 

adopting  and  tapping  commercial  supply  chain  management  best  practices,  the  entire  DoD 

weapons  system  sustainment  process  could  benefit.” 

One  of  the  PBL  success  boosters  of  Cebeci  (2009,  p.47)  is  working  with  experienced 

contractors.  The  experience,  he  mentioned,  is  not  only  PBL  experience,  but  also  work  experience 

of  provider  and  acquisition  team. 

“The  Department  of  Defense  shall  maintain  a  fully  proficient  acquisition,  technology,  and 
logistics  workforce  that  is  flexible  and  highly  skilled  across  a  range  of  management, 
technical,  and  business  disciplines.  To  ensure  this,  the  USD(AT&L)  shall  establish 
education,  training,  and  experience  standards  for  each  acquisition  position  based  on  the 
level  of  complexity  of  duties  carried  out  in  that  position.”  (DoDD  5000.01,  2003,  p.8) 

Customer  satisfaction  is  one  of  the  metric  that  can  measure  success.  Cebeci  (2009,  p.50) 

finds  out  that;  it  is  the  most  important  factor  among  all.  The  rating  of  PSI’s  customer  satisfaction 

will  improve  the  success.  Cebeci  (2009,  p.48)  identifies  customer  satisfaction  as  one  of  the  goals 

of  PBL. 


“This  method  of  contracting  utilized  for  years  in  many  aspects  of  production  equipment, 
facilities,  and  business  services,  is  becoming  more  prevalent  in  the  Aerospace  and 
Defense  (A&D)  Industry  as  the  Department  of  Defense  (DoD)  and  commercial  airline 
industry  try  to  decrease  lifecycle  costs  while  improving  end-customer  satisfaction.” 
(Newsome,  2008,  p.2) 

Cebeci  (2009,  p.  16)  cites  Garrett  (2005)  and  states  that  quality  of  the  staff  (for  both 
partners)  is  a  clear  simple  element  of  a  successful  PBL.  Manpower  and  personnel  requirements 
for  both  organic  and  commercial,  is  a  key  logistics  criteria  to  be  considered  (DoD  Designing  and 
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Assessing  Supportability  Guide,  2003,  p.25).  Especially  the  key,  critical  personnel’s  continuity 
should  be  committed  for  success.  Not  only  continuity  but  also  ongoing  education  and  training  are 
obligations  for  effective,  efficient  performance  (Sols  and  Venna,  2007,  p.48).  The  support 
strategy  should  consider  how  to  achieve  most  efficient  and  cost-effective  mix  of  manpower  from 
all  partners  (DoDD  5000.02,  2002,  p.38).  Trovato  (2004,  p.22)  defines  this  partnership  as  two 
sides  of  the  acquisition  coin.  This  partnership  shall  provide  best  practices  for  all  processes.  In 
Baldwin  et  al.’s  (2005,  p.33)  study,  “Interviewees  and  ISM  conference  participants  emphasized 
that  an  essential  element  of  any  purchasing  organization’s  current  and  future  success  is  the 
quality  and  expertise  of  its  personnel.”  According  to  Gansler,  et  al.  (2011,  p.  12-13)  military  is 
training  young  personnel,  meanwhile  commercial  especially  OEMs  have  the  expertise.  PBL 
gives  the  advantage  of  having  experience  at  a  given  short  time.  Therefore  it  is  easy  to  state  that 
one  of  the  potential  benefits  of  PBL  is  having  skilled  workforce. 

Vitasek  et  al.  (2006a,  p.8)  suggest  DoD  to  “press  ahead  with  vigor,  and  work  towards 
alleviating  any  remaining  financial,  statutory,  or  policy  barriers  that  limit  the  full  potential  of 
PBL”  and  that  will  result  utilizing  best  practices.  Also,  the  report  issued  by  PSAT  recommends 
DoD  to  “connect  platform  product  support  strategies  to  enterprise  supply  chain  approaches  that 
produce  best  value  across  the  DoD  components.”  (Fowler,  2010,  p.  16-17) 

Geary  et  al.’s  (2010,  p.475)  case  studies  came  up  with  several  changes  which  are  needed 
for  an  improved  PBL,  future  acquisition  strategies:  “Integrating  of  government  post-production 
support  capabilities  as  best-value  partners  into  a  unified  industrial  base”  and  “creating  the  correct 
blend  of  government  and  industry  partnership  based  on  best  value  capabilities,  not  statutory 
entitlement”  are  two  of  them. 
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Twenty  one  documents  explain  this  factor  with  examples.  Table  9  shows  the  examples  used 


by  corresponding  documents. 

Table  9.  Examples  used  in  documents  -  Best  practices. 


Example 

Reference 

Key  Points 

Typhoon 

BAE  (2010) 

Workforce:  Our  experienced  teams  work  side  by  side  with  RAF  personnel  to  deliver  aircrew  and  groundcrew  training,  maintenance 
and  servicing,  technical  support,  and  effective  inventory  and  asset  management. 

JSF 

Blumberg  (2006b) 

Experience:  Eaton  Aerospace  approaches  the  risk  with  a  base  comfort  because,  in  its  commercial  experience,  Eaton's  pumps  and 
actuators  demonstrate  high  reliability. 

Tornado 

Canaday (2006) 

Ability:  The  DLO  wanted  to  make  sure  BAE  had  the  resources,  skills,  capabilities,  facilities  and  ability  to  manage  the  project.  BAE 
beefed  up  its  resources  in  line  with  the  planned  strategy. 

JSF 

Canaday  (2006) 

Ability:  The  practical  ability  of  Defense  offices  to  switch  support  responsibilities  on  a  major  aircraft  will  require,  first,  adequate 
technical  data,  second,  the  right  skills  in  government  offices,  and  third,  competent  replacement  firms. 

Lockheed  Martin 
partnered  PBL  programs 

Cothran  (2008) 

Ability:  To  better  posture  themselves  for  this  growing  market,  they  have  instituted  a  comprehensive  Logistics  work  force 
development  effort  aimed  at  equipping  their  logistics  personnel  with  the  requisite  skills,  knowledge,  and  abilities  to  successfully 
accomplish  this  business  strategy. 

Defense  Transportation 
Coordination  Initiative 
(DTCI) 

Dibenedetto 

(2007) 

Best  practice:  Utilizing  best  commercial  practices  to  enable  load  consolidation  and  optimization,  use  of  more  efficient  intermodal 
means  of  transportation  and  tailored  cheduling  to  meet  the  customer  requirements. 

Intercontinental 

Ballistic  Missiles  (ICBMs) 

Goure  (2011a) 

Experience:  The  current  integrating  contractor,  TRW/  Northrop  Grumman,  has  some  60years  experience  in  designing,  managing, 
building  and  supporting  ICBMs.  Having  a  single  "button"  to  press  simplified  the  management  process,  allowing  the  Air  Force  to 
shrink  its  workforce  and  probably  reduced  costs.  This  system  has  worked  extremely  well. 

Intercontinental 

Ballistic  Missiles  (ICBMs) 

Goure  (2011b) 

Experience:  Were  we  dealing  with  fly-by-night  companies,  there  might  be  a  rationale  for  how  the  ALC's  sometimes  behave.  But  1  am 
talking  about  Pratt  &Whitney,  Northrop  Grumman  and  Boeing.  These  are  world  class  companies  with  decades  of  experience  in  both 
commercial  and  military  work,  great  supply  chain  management  skills  and  unique  engineering  talent. 

MCH-101 

Harada  (2010) 

Experience:  The  proposed  PBL  contractor,  Agusta  Westland,  produces  the  MCH-101  and  has  experience  with  the  weapon  system. 

FUR 

Vitasek  and 
Geary  (2008) 

Best  practice:  Each  party  was  able  to  utilize  their  best  practices  to  streamline  the  FLIR  repair  process. 

GE  404 

Vitasek,  et  al. 
(2006a) 

Workforce:  PBL  partnership  between  the  US  Navy  and  General  Electric,  leveraging  the  Navy's  workforce  repair  skills  with  the  engine 
OEM's  knowledge  of  the  production  process. 

JSF 

Vitasek,  et  al. 
(2006b) 

Workforce:  The  contractor  is  able  to  contract  back  with  the  government  for  skilled  labor  at  a  low  cost,  in  effect  making  the 
government  both  a  customer  and  a  supplier  to  the  contractor. 

As  a  result  best  value  providers  selected  among  industry  and/or  government  is  a  key 
element  of  successful  PBL  environment  (Kratz,  2001;  Sols  and  Venna,  2007).  PBL  is  the  tool 
leveraging  the  capabilities  of  partners  (Blumberg,  2006a,  p.57;  Blumberg,  2007a,  p.42;  Kratz  and 
Buckingham,  2010b,  p.299).  “Under  PBL,  private  sector  contractors  adapt  their  best  commercial 
practices  to  large-scale  program  management,  including  risk  mitigation  and  streamlining 
logistics  chains.”  (Blumberg,  2006b,  p.58)  “Effective  PBSA  is  considered  a  business  best 
practice  in  the  commercial  world.”  (Gansler,  et  al.,  2011,  p.57) 

8.  PBL  Experience,  Awareness  and  Training 

In  thirty  four  out  of  eighty  one  documents,  PBL  experience,  awareness  and  training  are 
mentioned  as  a  success  factor  for  PBL.  PBL  is  a  new  culture,  even  still  for  DoD.  All  DoD 
documents  have  a  section  regarding  the  implementation  of  PBL.  Lor  instance;  DoD  PBL  Guide 
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(2005)  introduces  a  method  for  implementing  a  PBL  product  support  strategy.  DoDD  5000.02 
(2002)  has  a  section  about  implementation  of  a  Performance-Based  Business  Environment. 
Memorandum:  Performance-Based  Services  Acquisition  (2000,  p.2)  has  an  attachment  “provides 
the  necessary  details  regarding  the  PBSA  implementation  plan,  training  requirements,  and  other 
Department-wide  PBSA  initiatives.”  In  addition  to  DoD  documents  some  of  the  authors  include 
implementation  processes  in  their  studies.  For  instance  Cebeci  (2009)  cites  Performance-Based 
Management  Process/Handbook  Model  (1997)  and  explains  each  step.  Berkowitz  et  al.  (2004) 
think  PBL  transition  is  a  culture  change.  According  to  them  organizational  members’  behaviors 
should  be  changed.  They  also  put  a  guideline  for  implementing  PBL  in  their  study.  Nicosia  and 
Moore  (2006)  lists  the  requirements  for  implementation  of  a  new  acquisition  system.  Vitasek  and 
Geary  (2007,  p.9)  explains  “a  twelve-step  process  that  it  recommends  for  implementing  PBL  and 
a  seven-step  process  for  implementing  Performance-Based  Services  Acquisition.” 

This  culture  change  is  a  challenge  for  all  partners.  “Most  of  the  personnel  and 
organizations  have  years  of  experience  developing  requirements-driven,  specification- 
constrained,  custom-designed  and  built,  components  and  systems.  For  many  of  the  DoD’s 
logistics  and  acquisitions  employees,  implementing  PPPs  changes  the  nature  of  their  work.” 
(Gansler  et  al.,  2011,  p.55) 

One  way  to  ease  the  implementation  of  PBL  is  experience.  Geary  (2006)  states  that 
experience  will  make  DoD  to  do  better  jobs.  “Maximizing  PBL  benefits  will  continue  to  drive  us 
to  develop  a  more  complete  understanding  of  the  risks  and  uncertainties  that  must  be  addressed, 
in  order  to  expand  adoption  and  meet  the  requirements.”  (Geary,  2006,  p.78)  The  more 
experience  gained  the  better  jobs  will  be  done  in  the  future.  Companies  who  have  experience  or 
emerging  best  practices  for  procuring  services  through  a  performance-based  approach  are  the 
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ones  to  be  studied  by  researchers  regarding  the  best  PBL  (Venema,  2007,  p.  1).  Even  mistakes 
will  make  you  gain  experience.  Cothran  says  that  “Companies  that  want  to  explore  a  PBL 
business  model  can  learn  much  from  the  mistakes  of  the  DoD”  (Vitasek  and  Geary,  2007,  p.8) 
Harada  (2010,  p.27)  suggests  the  government  to  look  how  commercial  firms  have 
successfully  implemented  performance-based  services  acquisition  to  have  a  better  understanding. 
Berkowitz  et  al.  (2004,  p.265)  recommends  “an  educational  program  to  clarify  the  understanding 
and  comprehension  of  the  definition,  scope,  and  purpose  of  PBL.” 

Acquisition  personnel  are  so  important  in  this  factor.  As  mentioned  before  Cebeci  (2009) 
acknowledges  that  “the  quality  of  an  acquisition  team  is  important.”  He  emphasizes  that  not  only 
‘experience  in  job’  is  enough  but  also  ‘experience  in  PBL’  is  required.  According  to  Kratz  and 
Buckingham  (2010a,  p.60)  “performance  based  arrangements  are  complex  and  require  a 
knowledgeable  DoD  life  cycle  workforce  that  has  core  competencies  in  all  product  support 
functions  and  full  insight/oversight  of  contract  and  agreement  execution.” 

As  stated  before,  PBL  may  still  be  counted  as  a  new  strategy.  Training  is  the  only  way  to 
be  aware,  sophisticated.  Training  is  counted  as  one  of  the  key  support  areas  in  DoD  PBL  Guide 
(2005). 

“A  report  of  the  Acquisition  Advisory  Panel  sums  it  up  best:  ‘When  individuals  without 
the  proper  training  and  experience  attempt  to  implement  a  performance-based  contract, 
the  results  are  understandably  and  expectedly  poor. .  .there  is  trouble  consistently 
implementing  it  by  an  inconsistently  trained  workforce.”  (Kobren,  2009,  p.265;  Vitasek 
and  Geary,  2008,  p.64) 

“Training  is  essential  to  increasing  performance-based  acquisition  for  services”  (Memorandum: 
Performance-Based  Services  Acquisition,  2000,  p.l;  Pettingill  et  al.,  2004,  p.2)  and  DoD  is 
“committed  to  providing  to  the  acquisition  workforce  the  training  and  tools  needed  to  define, 
acquire,  and  manage  service  requirements  efficiently  and  effectively.”  (Memorandum: 
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Performance-Based  Services  Acquisition,  2000,  p.l)  According  to  Kobren  (2010)  training  is  one 
of  the  critical  requirements  for  PBL.  Also  Geary  et  al.  (2010)  determines  training  and  education 
as  the  fundamental  action  for  PBL  implementation. 

There  are  several  certificate  courses  present  for  DoD.  According  to  Gansler  et  al.  (2011, 
p.56);  although  there  is  still  much  to  do,  “DoD  has  made  considerable  progress  on  designing  and 
implementing  appropriate  educational  and  training  programs.”  Brown  and  Cothran  (2005)  lists 
these  educational  options;  PBL  Training  Courses,  PBL  Continuous  Learning  Module,  PBL 
Online  Knowledge  Sharing,  PBL  Researches  by  Defense  Acquisition  University  (DAU).  DAU 
knows  what  is  needed: 

“PBL  contracting  has  demonstrated  we  can  apply  the  concept  of  performance-based 
outcomes  to  a  particular  sector  of  DOD  support.  We  need  to  inculcate  the  DOD  culture, 
using  lessons  learned  from  PBL,  moving  toward  perfonnance-driven  outcomes  in  all 
areas.  We  need  to  build  on  lessons  learned  from  continuous  process  improvements,  and 
apply  an  overall  perfonnance-based  outcomes  approach,  tying  all  budgeting  and 
contracting  to  a  warfighter-supported  metric.”  (Blumberg,  2007a,  p.42) 

Kobren  (2010,  p.199)  finds  these  courses  useful  for  “more  rigorous,  competency-based  defense 

acquisition  workforce.” 

Blumberg  (2006a,  p.60)  showed  the  gap  in  educating  the  contractor  side  and 
recommended  education,  especially  “executive  education  to  get  the  top  people  on  board  with  the 
concept”  due  to  this  culture  change.  According  to  Vitasek  and  Geary  (2007)  one  half  of 
education  is  education  of  DoD  personnel,  the  other  is  contractors.  Tennessee’s  Center  for 
Executive  Education  has  PBL  courses  for  service  providers.  Industry  leaders  such  as  Lockheed 
Martin,  Raytheon,  Pratt  Whitney,  Northrop  Grumman  and  Boeing  sponsored  and  signed  up  for 
these  courses.  Wuchenich  (2008)  explains  the  courses  in  his  study  and  states  that  one  of  the  four 
popular  research  subjects  of  University  of  Tennessee  is  PBL  training  programs.  As  a  result 
“there  are  two  main  resources — the  Defense  Acquisition  University  and  the  University  of 
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Tennessee.”  (Vitasek  and  Geary,  2008,  p.64)  In  addition  to  that  they  teamed  up  for  a 
collaborative  education  workshop.  “The  DoD's  Defense  Acquisition  University  group  teamed 
with  the  University  of  Tennessee  to  run  joint  workshops  for  DoD  program  managers  and  their 
contractors.”  (Maylett  and  Vitasek,  2008,  p.5)  These  are  the  things  done  for  education  and 
training  for  PBL. 

Devries  (2004)  identifies  lack  of  PBL  awareness  and  training  as  one  of  the  barriers  for 
PBL  implementation.  Half  of  the  survey  participant  programs  (26  programs)  identified  lack  of 
PBL  training  as  a  barrier.  According  to  his  research  results;  it  is  one  of  the  most  frequently 
encountered  barriers. 

DoD  wants  Program  Support  Managers  (PSMs)  to  have  “At  least  8  years’  acquisition 
experience,  which  includes  at  least  6  years  in  LCL,  with  at  least  2  years  in  a  program  office  or 
similar  organization.”  (Kobren,  2010,  p.197)  DoD  emphasizes  on  experience  especially  for  life 
cycle  support  programs  such  as  PBL.  Kobren  (2010,  p.199)  recommends  DoD  to  develop 
“highly  capable,  highly  effective  PSMs  with  a  truly  strategic,  enterprise-level  perspective.” 
Employees  dealing  with  PBL  should  “have  the  appropriate  training  and  tools  needed  to  do  their 
jobs.”  (Gansler  et  al.,  2011,  p.59) 

Blumberg  (2007a)  states  that  top  level  management’s  commitment  from  both  sides  is 

required  for  PBL  implementation.  Top-level  support  from  both  parties  is  one  of  the  success  keys 

of  PBL  according  to  Kate  Vitasek  (Blumberg,  2007b,  p.5). 

“Vitasek  says  you  need  champions  on  both  the  contractor  and  the  DOD  side  for  the 
business  model  to  shift  to  a  performance-based  model.  . . .  They  both  have  to  be  equally 
dedicated  to  dynamic  performance  approaches,  not  just  taking  an  existing  program  and 
slapping  on  metrics  and  a  bonus  structure”  (Blumberg,  2007a,  p.44) 

“Attaining  the  full  benefits  of  public-private  partnerships  can  only  be  accomplished  by 
continuing  to  work  to  change  the  culture  within  DoD — and  in  Congress — to  view  PPPs 
as  collaborations  that  can  bring  the  best  of  the  public  and  private  sector  knowledge  and 
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resources  to  bear  on  the  issue  of  sustaining  weapons  systems.  This  is  an  education 
process  that  must  address  several  perspectives.”  (Gansler  et  ah,  2011,  p.64) 

Nine  documents  give  examples  of  experience  and  training  for  PBL  implementation  and 

culture  change.  Table  10  shows  the  examples  used  by  corresponding  documents. 


Table  10.  Examples  used  in  documents-  PBL  Implementation,  Experience. 


Example 

Reference 

Key  Points 

NAVICP 

(A/C  Tires  and  APU) 

Berkowitz  et  al. 
(2003) 

Top  Management  Commitment:  Commanders  pushed  the  PBL  concept  through  the  organization.  Early  on,  the  leadership  recognized 
and  verbalized  the  benefits  of  PBL. 

Culture  Change:  Culture  champions  exist  throughout  DLA  to  devise  transformation  activities  to  close  gaps  in  the  culture  between 
today's  baseline  and  its  goal  of  becoming  a  truly  customer-focused  organization. 

F-22  Raptor 

Blumberg  (2006b) 

Training:  Alot  of  internal  company  PBL  training  is  done.  It  was  born  out  of  the  commercial  work.  It's  been  taken  a  one-company 
approach  across  divisions,  sharing  best  practices  between  commercial  and  military  operations. 

U.K.  PBL  Programs 

Canaday (2006) 

Experience:  BAE  Sunit  has  been  working  for  the  U.K.  on  a  series  of  pilot  programs  for  five  years.  In  December  2005,  the  Ministry  of 
Defense  announced  a  Defense  Industrial  Strategy  that  aims  for  through-life  capability  on  all  new  aircraft  acquisitions.  The  pilot 
programs  achieved  impressive  results  and  built  valuable  experience. 

Lockheed  Martin 
partnered  PBL  programs 

Cothran (2008) 

Experience,  Implementation:  The  Global  Hawk  program  office  is  aggressively  pursuing  PBL  initiatives.  Thus  far,  they  have  consulted 
with  JSTARS  representatives  and  have  developed  an  implementation  schedule  for  PBL  arrangement  by  2007.  It  is  identified  the 
comprehensiveness  of  the  Global  Hawk's  implementation  schedule  as  a  best  practice.  The  schedule  is  particularly  noteworthy 
because  it  presents  objectives  forthe  integration  of  PBLthroughout  an  8-year  period  (FY04-FY11). 

Navy  PBL  Programs 

Goure  (2010) 

Top  Management  Commitment:  One  of  the  several  reasons  why  the  Navy  has  had  such  success  is;  its  willingness  to  bring  all  the 
stakeholders  to  the  table  and  to  manage  the  effort  holistically. 

HI  MARS 

Lessons  Learned 
(2004) 

Experience  and  Training:  The  lesson  learned  is  that  experience  and/or  training  on  PBL  contracting  across  all  levels  (including  the 

Army  Staff)  is  vital  to  the  successful  award  and  implementation  of  PBL  contracts. 

H-60FLIR 

Lessons  Learned 
(2004) 

Experience:  Each  platform  has  unique  challenges  and  limitations  so  comparisons  of  one  program's  PBL  to  another  may  be 
misleading.  Additionally,  each  contractor  is  different  and  has  its  own  level  of  experience  and  confidence  in  the  PBL  process. 

F-22  Raptor 

Marietta  (2008) 

Implementation:  The  2008  PBL  System  Level  Award  went  to  the  F-22  team  .The  PBL  awards  program  recognizes  government  and 
industry  teams  that  demonstrate  outstanding  achievements  in  PBL  development,  implementation  and  execution.  This  is  the  third 
time  that  a  Lockheed  Martin  program  or  company  has  won  the  annual  award. 

Global  Hawk 

Pettingill  et  al 
(2004) 

Ability:  The  practical  ability  of  Defense  offices  to  switch  support  responsibilities  on  a  major  aircraft  will  require,  first,  adequate 
technical  data,  second,  the  right  skills  in  government  offices,  and  third,  competent  replacement  firms. 

As  a  result  PBL  is  a  culture  change  not  only  in  DoD  but  also  in  private  sector  (Gansler,  et 
ah,  2011).  “Success  requires  a  clear  understanding  of  the  organizational  changes  required, 
inclusion  of  partner.”  (Burkett,  2008,  p.5)  Experience,  awareness,  knowledge  of  PBL  is  a  success 
key  for  implementation.  Vitasek  and  Geary  (2008,  p.64)  identifies  one  of  the  fifteen  attributes  of 
a  good  PBL  is:  “Comprehensive  knowledge  and  experience  in  PBL  concepts,  tenets,  business 
model,  and  implementation  of  PBL  strategies.” 

9.  Innovation 

In  twenty  eight  out  of  eighty  one  documents,  it  is  stated  that  innovation  is  an  obligation 
for  PBL.  DoDD  5000.01  (2003)  mandates  using  PBL  strategies  whenever  feasible;  not  only  to 
increase  competition,  and  interoperability  but  also  innovation.  As  stated  before  providers  are  not 
to  be  told  how  to  do,  they  are  allowed  to  use  innovative  approaches.  These  statements  are  cited 
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from  Memorandum:  Performance-Based  Services  Acquisition  (2000)  and  also  used  in  DoD  PBL 

Guide  (2005).  Some  authors  such  as  Kratz  (2001),  Bozkurt  and  Guducu  (2005),  Cebeci  (2009) 

cites  DoD’s  statements  in  their  study,  to  emphasize  innovation.  Boehk  (2003,  p.l)  identifies 

DoD’s  PBL  strategy;  “an  innovative  new  program.” 

Perfonnance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide  (2005) 

explains  product  support  boundaries  (PSB)  and  even  these  boundaries  include  innovation. 

“PSB  provides  the  strategic  construct  for  innovation  within  a  consistent  support  structure 
for  the  warfighter.  ...  The  aim  of  the  PSB  is  to  provide  the  boundary  conditions  for 
product  support  strategies  that  allow  innovation  but  ensure  consistency  and 
interoperability  across  programs.”  (Perfonnance  Based  Logistics:  A  Program  Manager’s 
Product  Support  Guide  (2005,  p.1-3) 

DoD  Acquisition  System  shall  govern  innovation  as  one  of  the  policies: 

“Throughout  the  Department  of  Defense,  acquisition  professionals  shall  continuously 
develop  and  implement  initiatives  to  streamline  and  improve  the  Defense  Acquisition 
System.  MDAs  and  PMs  shall  examine  and,  as  appropriate,  adopt  innovative  practices 
(including  best  commercial  practices  and  electronic  business  solutions)  that  reduce  cycle 
time  and  cost,  and  encourage  teamwork.”  (DoDD  5000.01,  2003,  p.3) 

Memorandum:  Performance-Based  Services  Acquisition  (2000,  p.l)  states  that  PBSAs 

allow  “offerors  maximum  flexibility  to  attain  the  greatest  degree  of  innovation  and  creativity.” 

“There  is  more  room  for  innovation  and  creativity.”  (Sols  and  Venna,  2007,  p.40)  Buyer  is 

relying  on  contractor’s  knowledge,  experience  and  skills.  “The  contractors  are  given  the 

necessary  freedom  and  responsibility  to  innovate  and  apply  their  domain  knowledge  and 

experience,  within  a  context  of  financial  motivation  and  incentives.”  (Sols  and  Verma,  2007, 

p.49)  According  to  DoD  Designing  and  Assessing  Supportability  Guide  (2003,  p.7)  providers  are 

allowed  “to  offer  innovative,  cost-effective  solutions.” 

GAO  report  (2008b,  p.8)  states  that  one  of  the  intents  of  PBL’s  benefits  is  “encouraging 

contractors  to  be  innovative  and  to  find  cost-effective  ways  of  delivering  services  for  a  fixed 
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level  of  funding.”  To  gain  that  benefit  focus  should  be  shifted  from  process  to  results. 

Buyukgural  (2009,  p.14)  cites  Rievley  (2001)  regarding  telling  contractors  how  to  do;  “limiting 

them  by  mandating  how  to  produce  their  product  or  service  under  non-PBL  contracts  may  result 

in  losing  an  opportunity  of  benefitting  from  their  innovative  approaches  and  cumulated 

experience.”  According  to  him  PBL  will  keep  its  promises  of  sustaining  desired  outcomes  with 

lower  costs  by  innovation.  “Performance-based  acquisition  creates  a  new  acquisition 

environment  for  both  industry  and  government  by  promoting  flexibility  and  innovation  and  by 

creating  win-win  solutions  through  effective  communication.”  (Cebeci,  2009,  p.54) 

“There  are  savings  to  be  had,  as  well  as  enhanced  performance.  Particularly  important  is 
the  ability  of  innovative  partnering  solutions  to  reduce  the  amount  of  work  in  progress, 
lower  the  burden  of  oversized  inventories  and  slow  supply  chains,  and  increase  the 
availability  of  repaired,  refurbished  and  reset  weapons  systems.”  (Goure,  2010,  p.2) 

Innovation  is  one  the  keys  for  the  promises  of  PBL.  PBL’s  goal  is  having  improved  performance 

while  decreasing  costs. 

“American  industry  provides  a  source  of  innovation,  and  flexible  and  productive  capacity 
for  the  defense  industrial  base.  The  way  ahead  for  more  cost-effective  product  support 
lies  in  effective  blending  of  these  complementary  capability  sets  where  the  best  use  is 
made  of  the  entire  industrial  base,  facilitated  by  the  continuing  expansion  of  best  business 
practices  in  both  the  commercial  and  government  sectors.”  (Geary  et.  al,  2010,  p.454) 

Goure  (2009)  sees  PBL  as  an  innovative  refonn.  Berkowitz  et  al.  (2004)  state  that; 

development  and  implementation  of  PBL  include  some  form  of  creativity,  innovation.  Aviation 

Week  Executive  Roundtable  participants  agree;  in  PBL  environment  much  innovation  and 

creativity  takes  place  (Hedden,  2005,  p.3).  PSI  should  have  creative  approach  to  find  great  ways 

to  improve  processes.  Innovative  approaches  are  required  for  a  better  PBL  (Vitasek,  2007). 

According  to  Kratz  and  Buckingham  (2010a,  p.59);  “industry  invests  in  development  costs  with 

an  equal  emphasis  on  maturation  and  innovation.”  Future  acquisition  strategies  will  include 

innovation  (Kratz  and  Buckingham,  2010a,  p.58;  2010b,  p.293). 
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Indeed,  DoD  needs  “innovative  ways  to  contain  costs  while  improving  the  perfonnance 
of  its  maintenance  and  logistics  system.”  (Goure,  2010,  p.l)  DoDD  5000.02  (2002,  p.31)  counts 
“the  use  of  pilot  programs  to  explore  innovative  practices”  in  best  practices.  Berko  witz  et  al.’s 
(2003,  p.  13)  study  shows  that  “in  1998  DoD  established  thirty  sustainment  pilot  programs,  of 
which  twenty-four  adopted  innovative  product  support  strategies.”  According  to  them,  new 
acquisition  systems  should  be  exceptional;  include  and  encourage  innovation  applying  ingenuity. 
The  underlying  idea  under  PBL  is:  “to  allow  contractor  to  apply  ingenuity  and  innovation  to 
efficiently  deliver  the  requirement  instead  of  dictating  the  Government  preferred  approach.” 
(Berkowitz  et  al.,  2003,  p.49) 

Geary  et  al.  (2010,  p.475)  list  “creating  a  culture  of  high-performing,  innovation-driven 
government-  industry  teams”  in  required  changes  and  their  study  concluded  that  depending  on 
the  case  analysis. 

Kelman  (2007)  emphasizes  the  importance  of  innovation  via  complaining  about  inspector 
generals.  According  to  him  they  are  not  focusing  on  creativity,  innovation.  Their  only  purpose  is 
controlling,  not  developing  creative,  innovative  solutions  to  problems. 

The  House  Armed  Services  Committee  (HASC)  Panel  on  Defense  Acquisition  Reform  in 
March  2009  recommends  “improving  competition  and  access  to  more  innovative  technology  by 
utilizing  more  of  the  industrial  base,  especially  small  and  mid-tier  businesses.”  (Kratz  and 
Buckingham,  2010b,  p.302) 

Eight  documents  supported  innovation,  as  a  factor  of  PBL’s  success,  with  examples. 

Table  1 1  shows  the  examples  used  by  corresponding  documents.  All  examples  are  candidates  of 
best  PBL  and  all  include  innovation  somewhere  and  somehow.  As  a  result  innovation  is  a  key  for 
a  successful  PBL. 
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Table  11.  Examples  used  in  documents-  Innovation. 


Example 

Reference 

Key  Points 

U.K.  PBL  Programs 

BAE  (2010) 

BAE  partners  with  the  UK  Royal  Navy  to  deliver  24/7  dedicated  support  to  a  number  of  the  UK's  warships,  working  to  ensure  they  are 
available  whenever  and  wherever  they  are  needed.  This  innovative  approach  to  support  the  UK's  River  Class,  Survey  Vessels  and 

HMS  Clyde  has  delivered  unprecedented  vessel  availability.  BAE  believes  that  the  have  skills,  creativity,  innovation,  experience  and 
know  how  to  proactively  find  bold  and  efficient  ways  to  deliver  value  for  our  customers. 

C-17 

Barnes  and 
Johnson  (2010) 

The  group's  mission  is  to  provide  executive  oversight  related  to  key  strategic  issues  impacting  the  C-17  depot  sustainment  activities 
and  product  support.  The  gatekeepers  help  determine  the  best  integrated  life  cycle  management  strategy  according  to  statutory 
and  regulatory  guidance.  In  addition,  they  provide  strategic  direction  to  establish  effective,  affordable,  and  innovative  strategies 
focused  on  C-17  sustainment. 

NAVICP  Programs 
(A/C  Tires,  APU) 

Beilis  (2001) 

These  innovative  government/industry  partnerships  has  revolutionized  how  NAVICP  provides  logistics  suppport  to  its  fleet 
customers. 

APU  -  Honeywell 

Gansleret  al. 
(2011) 

Better  forecasting  and  the  other  innovations  results  lots  of  cost  savings. 

Hunstville  (ITAS, 

HI  MARS,  Shadow  2000, 
TAIS) 

Geary  et.  al 
(2010) 

Team  climate  for  innovation  and  team  innovation  are  two  of  the  eight  critical  PBL-driven  interorganizational  success  factors.  They 
have  a  culture  of  demonstrating  innovation  and  leadership  when  it  comes  to  post-production  support.  They  created  an  innovative 
environment  where  the  adoption  and  application  of  best  commercial  business  practices  have  been  embraced. 

C-130J 

Geary  et.  al 
(2010) 

The  stratgy  implemented  for  a/c  is  innovative  and  successful.  This  innovation  sets  a  benchmark  for  partnering  with  industry  and 
leverages  that  relationship  to  increase  weapons  system  availability  while  reducing  operating  costs. 

C-130  Avionics 

Modernization 

Kratz  (2001) 

Based  on  the  lessons  learned  from  the  pilot  programs,  new  weapon  systems  and  major  upgrade  programs  are  implementing  a 
variety  of  innovative  PBL  strategies  that  capitilize  on  effective  government/industry  partnerships. 

Army  Sentinel 

Kratz  (2001) 

The  system  will  employ  an  innovative  PBL  strategy.  Approach  features  life  cycle  system  support  through  PSI  including  integrated 
logistics,  total  asset  visibility,  depot  partnering  and  unified  configuration  management. 

F-22  Raptor 

Marietta  (2008) 

F-22  Raptor  as  an  exemplary  program  in  Performance  Based  Logistics  (PBL),  an  innovative  support  approach  that  provides  higher 
aircraft  readiness  to  warfighters  and  lower  overall  costs  to  taxpayers. 

F-117 

Marietta  (2008) 

Lockheed  Martin  won  the  System  Level  Award  as  part  of  the  Air  Force  F-117  Nighthawk  total  system  support  partnership  team,  for 
innovative  contracting  measures  enhancing  warfighter  capabilities  and  cutting  costs. 

JSTARS 

Slinkard  and 
Poleskey  (2006) 

Contracting  approach  allowed  both  government  and  contractor  to  appropriately  share  the  risk  associated  with  the  firm  pricing  out 
year  requirements  within  the  pottential  award  tern  period.  Similar  innovative  approaches  can  also  be  applied  to  the  competitive 
environment. 

10.  Data  Requirement 

In  twenty  seven  out  of  eighty  one  documents,  it  is  stated  that  data  is  a  requirement  for 
PBL.  There  are  three  kinds  of  data  needed;  historical  data  for  baseline,  cost  data  for  estimations 
and  present  data  for  evaluations. 

PBL  is  invented  especially  for  providing  improved  performance.  According  to  DoD  PBL 
Guide  (2005)  perfonnance  baselines  may  be  calculated  from  past  experience.  It  is  “very  hard  to 
guarantee  product  availability  due  to  significant  uncertainties  in  product  reliability  and  usage  as 
well  as  inherent  product  complexity,  resulting  in  large  risks  to  both  the  customer  and  service 
provider.”  (Cohen  and  Netessine,  2007,  p.2)  Performance  Based  Logistics:  A  Program 
Manager’s  Product  Support  Guide  (2005,  p.3-28)  forces  BCA  to  include  data  required  to  support 
and  justify  the  PBL  strategy,  and  data  sensitivity. 
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According  to  Gardner  (2008,  p.48-50)  lack  of  the  historical  data  is  one  of  the  factor 

influences  PBL  strategy.  “Both  the  government  and  commercial  vendors  must  consider  various 

risks  such  as  . . .  accuracy  of  future  demand  based  on  historical  data.” 

Also  for  accurate  evaluation;  a  baseline  is  needed.  “System  perfonnance  is  to  be 

evaluated  against  a  performance  baseline  based  on  historical  data.  . . .  Historical  perfonnance 

data  are  essential  in  setting  sound  objectives  in  PBL  contracts.”  (Sols  and  Venna,  2007,  p.47) 

Aviation  Week  Executive  Roundtable  participants  define  “the  appropriate  level  of 

system/subsystem/item  to  be  included  in  a  performance-based  contract”  as  a  challenge  according 

to  data  requirements  (Hedden,  2005,  p.3).  Historical  data  is  also  a  requirement  for  processes  of 

support.  “The  collection,  analysis,  and  evaluation  of  system  performance  and  maintenance 

perfonnance  data  to  determine  the  need  for  and  prescribe  changes  to  the  system  configuration, 

maintenance  support  structure,  and  maintenance  resource  requirements”  is  a  key  logistics  criteria 

according  to  DoD  Designing  and  Assessing  Supportability  Guide  (2003,  p.25). 

Buyer  must  have  the  ability  to  reach  the  data  at  any  given  time.  In  case  of  applying  a 

system  level  PBL,  “customer  often  lacks  maintenance  and  logistics  data  to  analyze  PBL.  ...  It  is 

not  unreasonable  to  assume  that  suppliers  would  charge  significant  sums  of  money  to  provide 

such  data  if  they  enjoyed  a  monopoly  on  such  information.”  (Harada,  2010,  p.23) 

“Data  Management  (DM)  is  an  important  part  of  Total  Life  Cycle  Systems  Management 
(TLCSM),  and  PBL  and  should  be  considered  early  in  the  acquisition  life  cycle.  Data 
systems  supporting  acquisition  and  sustainment  should  be  connected,  real-time  or  near 
real-time,  to  allow  logisticians  to  address  the  overall  effectiveness  of  the  logistics  process 
in  contributing  to  weapons  system  availability  and  Life  Cycle  Cost  (LCC)  factors.” 
(Perfonnance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide,  2005,  p.4- 
28) 

Maj .  Gen.  Daniel  Mongeon  thinks  logistics  data  as  DNA  building  blocks  for  acquisition. 
According  to  him  data  integration,  interoperability  and  integrity  are  all  key  factors  of  success 
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(Anonymous,  2005,  p.33).  “Continuous  and  accurate  infonnation  enables  the  PBL  contractor  to 
anticipate  demand,  identify  and  implement  desirable  change  in  design,  fabrication  or 
transportation  of  items,  and  even  alternative  maintenance  practices.”  (Gansler  et  ah,  2011,  p.14) 
Nicosia  and  Moore  (2006)  thinks  improving  data  availability,  quality,  and  utilization  are 
important  for  both  government  and  industry.  It  is  used  not  only  for  performance  evaluation  but 
also  for  supplier  selection. 

Baldwin  et  al.  (2005,  p.  xiii)  recommends  DoD  (especially  Air  Force)  “to  implement  new 
data  collection  procedures  for  many  of  the  required  data,  particularly  supplier  and  customer 
satisfaction  data.”  Kratz  and  Buckingham  (2010a,  p.61)  states  that  “Government  should  procure 
access  and  rights  to  system  technical  data  to  enable  long-term  sustainment  and  competition” 
regarding  DoD’s  PBL  efforts.  PSAT  reports  suggests  so:  “Develop  an  overarching  DoD 
sustainment  metrics  and  management  strategy  for  life  cycle  product  support  that  strengthens 
formal  data  collection  and  analysis  capabilities  while  providing  insight  and  learning  to  support 
life  cycle  planning  and  operational  management.”  (Fowler,  2010,  p.  17;  Kratz  and  Buckingham, 
2010b,  p.298) 

Blumberg  (2006b;  2007a)  states  that  at  least  two  years’  data  (such  as  production  data  and 

reliability  data)  is  needed  to  apply  PBL  to  a  legacy  program. 

“A  large  defense  prime  contractor  summed  it  up  well:  PBL  contracts  demand  more  data 
to  be  successful.  It’s  necessary  to  mine  that  data  deeper,  get  cost  roll-ups  on  total  systems 
versus  components,  identify  redundancy,  analyze  mean-time-between-failure  reliability 
data,  and  optimize  spares  and  service  technician  placement.”  (Burkett,  2008,  p.5) 

As  a  result  regarding  historical  and  current  data;  “A  successful  perfonnance-based 

agreement  (PBA)  generally  requires  a  continuous  flow  of  high-quality  information  about  the 

status  and  history  of  every  element  of  the  supply  chain  and  about  the  parts,  systems,  or  even 

platfonns  subject  to  the  contract.”  (Gansler  et  al.,  2011,  p.14) 
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Not  only  perfonnance  data  but  also  cost  data  is  required.  “The  cost  and  performance 

baselines  for  legacy  systems  will  be  determined  by  historic  experience  and  costs.  The  cost 

baseline  will  include  all  appropriate  Government  and/or  contractor  costs,  including  indirect 

costs,  overhead,  and  handling  fees.”  (Performance  Based  Logistics:  A  Program  Manager’s 

Product  Support  Guide,  2005,  p.3-29)  DoDD  5000.02  (2002,  p.31)  lists  “realistic  cost  estimates 

and  cost  objectives”  under  best  practices.  Obviously  data  is  required  for  that. 

It  is  really  hard  to  detennine  a  cost  for  a  service  when  there  is  not  any  market  for  that. 

Doerr  et  al.  (2005,  p.  1 65)  describe  this  with  a  generic  example: 

“. . .  when  discussing  the  transportation  of  freight  within  the  U.S.,  prices  are  perhaps  not 
difficult  to  detennine  by  reference  to  a  market  for  commercial  transportation  services. 
However,  when  discussing  a  service  such  as  intermediate-level  maintenance  of  a 
deployed  weapon  system  such  as  an  aircraft,  on  which  the  DoD  has  a  monopsony  and  the 
number  of  qualified  bidders  is  quite  limited  (and  may  indeed  be  only  one  or  two),  the 
market  paradigm  clearly  breaks  down,  and  market  prices  are  not  available.” 

Especially  for  new  systems,  DoD  chooses  Cost  Plus  contracting  approaches  at  the 

beginning.  “As  a  general  rule,  Fixed  Price  contracts  should  be  avoided,  until  the  price  can  be 

predicted  at  a  certain  level  of  confidence.”  (Bozkurt  and  Guducu,  2005,  p.14)  We  do  not  know 

the  cost  at  all  at  early  phases.  Although  it  is  not  hard  for  legacy  systems  due  to  having  lots  of 

data,  sometimes  it  may  be  a  problem,  too. 

“Even  assessing  what  it  really  costs  the  government  to  perfonn  logistical  support  for  a 
weapon  system  is  often  a  hugely  daunting  task.  It  typically  requires  analysis  of  years  of 
data,  and  to  be  realistic,  must  analyze  a  myriad  of  costs  that  are  hidden  throughout  an 
enormous  bureaucratic  system.”  (Gardner,  2008,  p.21) 

GAO  Report  (2008b)  emphasizes  on  cost  data  requirement  especially  in  business  case 

analysis.  One  of  the  recommendations  to  DoD  in  GAO  Report  (2008b,  p.6)  is: 

“Require  the  development  of  business  case  analyses,  specifying  the  elements  to  be 
included  in  them  so  that  they  are  comprehensive  and  sound  and  that  the  services  improve 
their  internal  controls  to  ensure  that  the  analyses  are  performed,  and  that  program  offices 
collect  and  report  detailed  support  cost  data  for  their  PBL  arrangements.” 
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Gardner’s  (2008,  p.50)  study  finds  out  that;  “various  risks  such  as  changes  in  costs  and 
operational  environments,  unknown  costs  associated  with  infant  programs,  and  accuracy  of 
future  demand  based  on  historical  data”  should  be  considered  by  both  partners  of  PBL. 

Seven  documents  give  examples  about  data  requirement.  Table  12  shows  the  examples 
used  by  corresponding  documents. 

According  to  Newsome  (2008,  p.4);  “the  ability  to  price  the  contract  correctly  (prior  to 
execution)”,  “determining  an  accurate  forecast  of  customer  requirements”  and  “understanding 
and  gaining  access  to  customer  requirements”  are  three  of  the  five  biggest  challenges.  These  are 
all  related  to  data  requirement. 


Table  12.  Examples  used  in  documents-  Data  Requirement. 


Example 

Reference 

Key  Points 

Army  Aviation  Missile 
Command  PBL  Programs 

Berkowitz  et  al. 
(2003) 

Cost  Data  Requirement:  The  study  identified  six  major  gaps  between  AMCOM's  business  operation  and  the  best  practices  from 
Defense  and  industry.  One  of  the  need  is  undertanding  cost  fully. 

U.S.  Army/AAI 

Unmanned  A/Cs 

Blum  berg  (2007a) 

The  University  of  Tennessee  case  study  points  out  that  Phase  1  of  the  U.S.  Army/AAI  contracting  relationship  (2003-2006)  allowed 
time  "to  validate  and  verify  the  metrics  and  data  collection  processes. 

JSF 

Canaday (2006) 

According  to  him  regarding  the  success  of  this  program;  full  PBL  needs  four  kinds  of  information:  1)  business  processes  for  handling 
needed  parts  and  repairs  2)  information  on  fleet  performance;  3)  data  on  parts,  their  configuration  and  location,  and  4)  performance 
data  and  interpretation  to  flag  the  need  for  parts  or  repairs  in  a  timely  fashion. 

C-17 

Hedden  (2007) 

The  C-17  support  team,  for  instance,  has  the  data  and  experience  needed  to  plan  and  anticipate  the  aircraft's  needs  in  a  high  tempo 
operation  environment.  Experience,  then,  becomes  an  asset.  Integrated  Defense  Systems,  the  focus  is  translating  the  KPPs  into 
metrics  that  companies  can  fully  define  based  on  data  generated  during  real-time  use,  and  then  pushing  those  metrics  throughout 
the  supply  chain. 

Navy  Aircrew  Survival 
Equipmentman 

Lessons  Learned 
(2004) 

It  is  recommend  starting  with  a  cost  contract  initially  to  establish  historical  basis/trend  data  for  growing  into  a  realistic  fixed-price 
environment. 

Global  Hawk 

Petti  ngi  II  et  al 
(2004) 

Authors  identified  the  comprehensiveness  of  the  Global  Hawk's  implementation  schedule  as  a  best  practice.  PM  started  considering 
PBL  as  a  support  strategy  early  in  FY04  by  defining  the  metrics,  costs,  and  the  systems  used  to  collect  this  data. 

Norwegian  Intercity  Rail 
(Industry  example) 

Sols  and  Verma 
(2007) 

Performance  and  Cost  Data  Requirement:  The  authors  emphasize  the  need  forthe  model  to  establish  a  reference  point  oryearand 
to  calibrate  a  cost  function  and  a  demand  function  during  this  reference  year.  A  historical  performance  basis  is  essential  to  setting  a 
sound  PBL  contract. 

Consequently  it  can  be  easily  stated  that  without  sufficient  data,  information  a  PBL  does 
not  work  well,  whether  the  system  is  new  or  legacy.  Therefore  data  is  required  for  a  successful 
PBL. 


“It’s  difficult  to  do  a  true  PBL  contract  if  you  don’t  have  good  data,  because  early  in  the 
lifecycle,  you  can’t  project  repair  costs  out  10  years.  You  just  don’t  have  the  history  of 
how  the  system  performs  in  the  field  over  time  and  usage.  ...  Asa  weapon  system 
matures  and  becomes  a  legacy  system,  you  then  look  at  pulling  some  of  the  major 
systems  or  subsystems  off  and  giving  them  to  the  OEM  to  manage.  You  would  now  have 
plenty  of  perfonnance  history  data  with  which  to  forecast  demand  and  manage  repair 
cycles.”  (Gansler  et  ah,  2011,  p.27-28) 
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11.  Competition  and  PSI  Candidates 

In  twenty  two  out  of  eighty  one  documents  competition  and  PSI  candidates  are  stated 

important  for  PBL.  Performance  Based  Logistics:  A  Program  Manager’s  Product  Support  Guide 

(2005)  identities  the  purposes  of  using  performance  based  strategies;  one  of  them  is  maximizing 

competition.  It  is  stated  so,  in  Memorandum:  Performance-Based  Services  Acquisition  (2000) 

and  DoDD  5000.01  (2003).  Not  only  DoD  documents  but  also  FAR  mandates  competition.  “In 

order  to  maintain  a  competitive  environment,  industry  participation  will  be  detennined  in 

accordance  with  the  FAR.”  (Performance  Based  Logistics:  A  Program  Manager’s  Product 

Support  Guide,  2005,  p.3-30)  Competition  is  one  of  the  acquisition  policies  of  DoD.  DoDD 

5000.01  (2003,  p.5)  explains  the  policy  in  details: 

“Competition  shall  provide  major  incentives  to  industry  and  Government  organizations  to 
innovate,  reduce  cost,  and  increase  quality.  All  of  the  DoD  Components  shall  acquire 
systems,  subsystems,  equipment,  supplies,  and  services  in  accordance  with  the  statutory 
requirements  for  competition.  Acquisition  managers  shall  take  all  necessary  actions  to 
promote  a  competitive  environment,  including  the  consideration  of  alternative  systems  to 
meet  stated  mission  needs;  structuring  Science  and  Technology  (S&T)  investments  and 
acquisition  strategies  to  ensure  the  availability  of  competitive  suppliers  throughout  a 
program’s  life,  and  for  future  programs;  ensuring  that  prime  contractors  foster  effective 
competition  for  major  and  critical  products  and  technologies;  and  ensuring  that  qualified 
international  sources  are  pennitted  to  compete.  If  competition  is  not  available,  PMs  shall 
consider  alternatives  that  will  yield  the  benefits  of  competition.” 

DoDD  5000.02  (2002,  p.31)  lists  “adequate  competition  among  viable  offerors”  under  ‘best 

practices’  title.  According  to  that  directive  competition  is  the  elements  that  support  strategy 

should  include.  It  is  the  most  powerful  incentive  in  the  industry. 

One  of  the  most  important  things  for  competition  is  willingness.  Buyer  should  find 

contractors  who  are  willing  to  accept  the  PBL  strategy.  “There  must  be  a  vendor  who  is  willing 

to  contract  with  the  customer.”  (Harada,  2010,  p.26)  In  case  of  not  having  a  willing  candidate, 

how  PBL  works.  According  to  Doerr  et  al.  (2005,  p.  1 69)  “the  best  PBL  candidates  are  those  with 
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external  markets  for  services.”  Service  should  have  a  market  at  least  for  parts.  “Granted,  not 
every  supplier  has  been  thrilled”  by  PBL  (Cohen  and  Netessine,  2007,  p.3).  Their  study  show 
that  5  of  the  128  suppliers,  who  are  solicited  by  DoD  at  a  recent  PBL  support  contract, 
responded.  But  suppliers  are  getting  used  of  PBL  and  getting  more  interested  day  by  day. 

Competition  is  fundamental  for  every  acquisition.  PBL  is  using  perfonnance 
requirements  to  “increase  the  opportunity  to  maximize  competition  on  supplier  base.”  (Bozkurt 
and  Guducu,  2005,  p.  19)  According  to  Bozkurt  and  Guducu;  it  is  one  of  the  main  objectives  of 
PBL. 


“The  most  likely  candidates  for  the  integrator  role  are: 

•  The  system’s  original  equipment  manufacturer  or  prime  contractor; 

•  An  organic  agency,  product,  or  logistics  command  (e.g.,  DLA,  Naval  Inventory 
Control  Point  (NAVICP),  depots); 

•  A  third-party  logistics  integrator  from  the  private  sector; 

•  The  PM’s  own  logistics  organization.”  (Perfonnance  Based  Logistics:  A  Program 
Manager’s  Product  Support  Guide,  2005,  p.3-12;  Pettingill  et  al,  2004,  p.9;  Gansler  et 
al„  2011,  p.7-8) 


Cothran  (n.d.,  p.9-10)  makes  a  similar  list  for  candidates  and  explains  them  in  details: 

“•  Prime  Vendor/Original  Equipment  Manufacturer  (OEM).  ...  The  Prime  Vendor/OEM, 
with  responsibility  for  designing,  producing,  and  successfully  fielding  a  subject  system 
has  a  vast  array  of  system  knowledge  and  corresponding  robust  infrastructure  (equipment 
and  facilities),  along  with  in-place  sub-contractor  support,  trained  personnel,  technical 
data,  proprietary  rights,  and  numerous  other  irreplaceable  qualities  and  skills  that  make 
them  eminently  qualified  to  assume  the  PSI  role. 

•  Third  Party  Logistics  (3PL)  Provider.  The  use  of  3PLs  is  becoming  more  prominent  in 
both  the  public  and  private  sectors.  3PLs  are  attractive  PSI  candidates  when  they  meet. . . 
significant  expertise  in  a  Logistics  functional  area  encompassed  by  the  PBL  relationship 
. . .  significant  experience  in  integration  management,  especially  when  there  is  no  clear 
‘Prime’  vendor/OEM. 

•  Organic  DoD  Organization.  For  legacy  systems  . . .  where  PBLs  will  most  often  be 
initiated  at  ‘less  than  system’  level,  the  overall  PSI  top-level  integration  function  will 
usually  be  done  organically,  either  directly  through  the  Program  Office,  or  in  partnership 
with  a  key  organic  support  organization,  such  as  a  Depot  or  Inventory  Control  Point. 

•  Program  Management  Office.  As  reflected  under  ‘Scope’,  above,  the  Product  Support 
Integration  function  may  be  accomplished  within  the  PMO  when  a  horizontal  PBL 
strategy  composed  of  both  PBL  and  non-PBL  discrete  a  functional  support  strategy  has 
been  implemented.” 
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Trovato  (2004)  lists  them,  too  and  states  that  those  are  all  organized  to  provide  what  is 
promised  by  PBL  via  competition;  higher  performance,  lower  costs.  Gansler  et  al.  (2011,  p.59) 
states  so:  “The  key  is  leveraging  competitive  pressure  to  obtain  better  perfonnance  at  a  lower 
cost.” 

Due  to  their  knowledge,  familiarity,  experiences  regarding  the  system,  OEMs  have  an 

advantage  as  mentioned  before.  They  have  advantage  in  sole  source  environments,  too. 

“. . .  most  PBLs  have  been  awarded  in  what  is  effectively  a  “sole  source  environment,” 
expressing  that  in  reality,  no  firm  other  than  the  prime  contractor  or  OEM  has  the 
knowledge  and  ability  to  provide  PBL  support  for  a  weapon  system,  so  “OSD’s  notions 
of  competition  are  really  not  operable.”  (Gardner,  2008,  p.66-67) 

In  most  cases  government  should  rely  on  OEMs,  because  of  not  having  another  candidate  or 

organic  support.  Gardner  (2008)  lists  this  phenomenon  in  his  ‘issues,  influencing  the  contracts’ 

table.  Berkowitz  et  al.  (2003,  p.50)  agree  that;  “OEM  has  an  advantage  early  in  the 

development/fielding  cycle  of  the  system”  and  “technology  insertion/obsolescence  is  best 

managed  by  the  OEM.”  Although  they  have  some  advantages,  DoD  does  not  favor  any 

candidate.  DoD  favors  knowledge.  The  contactor  “having  the  most  intimate  knowledge  of 

manufacturing  processes,  system  reliability,  and  potential  improvements,  may  be  a  prime 

candidate  for  entering  into  a  public/private  teaming  relationship.”  (Perfonnance  Based  Logistics: 

A  Program  Manager’s  Product  Support  Guide,  2005,  p.3-12) 

“While  a  majority  of  successful  PBL  Product  Support  Integrators  (PSI)  are  in  fact 
industry  partners  (and  in  many  cases,  the  OEM),  contrary  to  popular  misconception,  there 
is  no  mandate  in  DoD  policy  to  use  a  commercial  sector  PSI,  or  even  use  an  industry 
product  support  provider.”  (Kobren,  2009,  p.260) 

DoD  should  “'select  best-value,  long-tenn  product  support  providers  and  integrators  based  on 

competition.”  DoD’s  “source  of  support  decisions  shall  foster  competition  throughout  the  life  of 

the  system.”  (DoDD  5000.02,  2002,  p.35-36;  Kratz,  2001,  p.14)  According  to  Kobren  (2009) 


78 


DoD  does  not  change  the  selection,  reliance  on  suppliers;  the  only  change  is  how  they  do 
business,  processes. 

PBL  is  good  to  be  used  where  the  candidates  are  hard  to  manage.  Harada  (2010)  agrees 

and  explains  it  with  Navy  example.  ‘Fewer  suppliers’  is  easier.  Nicosia  and  Moore  (2006) 

believe  that  with  few,  transactions  go  down,  manageability  increases;  but  on  the  other  hand  ‘few 

by  means  lean’  has  risks.  Buyer  should  pay  attention.  Baldwin  et  al.  (2005,  p.  1 3)  agree  that: 

“. . .  determining  the  ‘right’  number  of  suppliers  for  the  company  could  mean  trying  to 
decrease  or  increase  the  number  of  suppliers  providing  a  given  good  or  service.  A 
company  with  too  many  suppliers  might  not  have  sufficient  leverage  over  any  individual 
supplier  to  reduce  costs  or  increase  performance.  On  the  other  hand,  a  company  with  too 
few  suppliers  could  be  at  risk  if  suppliers  do  not  feel  competitive  pressure  to  innovate  and 
improve  or  if  suppliers  have  difficulty  fulfilling  their  commitments.” 

One  of  the  top  ten  challenges  is  management  of  suppliers  as  Newsome  (2008,  p.5)  stated. 

“A  support  environment  that  maintains  long-tenn  competitive  pressure”  is  a  key  element 
of  PBL  (Kratz,  2001,  p.12).  According  to  Kratz  (2001)  pilot  programs  thought  that  buyers  must 
take  necessary  actions  to  sustain  long  term  competition. 

Kratz’s  and  Buckingham’s  (2010a;  2010b)  list  of  future  acquisition  strategies  have 
industry  competition  as  a  property  in  common.  Therefore  it  is  a  basis  to  be  paid  attention  for 
acquisition  strategies.  Although  U.S.  Government  believes  that  PBL  is  decreasing  competition 
according  to  the  long  term  contracts;  Goure  (2010)  emphasized  the  importance  of  industry 
competition  in  acquisition  and  argues  U.S.  Government  statements  regarding  the  declared  reason 
for  going  back  from  PBL  is  to  have  more  competition. 

HASC  Panel  on  Acquisition  Reform  recommends  DoD  to  improve  competition  all¬ 
business-wide  and  throughout  the  life  cycle  of  a  system  (Kratz  and  Buckingham,  2010b,  p.301). 
DoD’s  life  cycle  management  suggests  and  mandates  to  do  so  for  all  stages  of  PBL. 
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Four  documents  used  examples  to  express  their  support  to  the  ideas  above.  Table  13 
shows  the  examples  used  by  corresponding  documents. 

As  a  result;  from  traditional  approaches  to  future  strategies  including  PBL,  competition  is 
a  requirement  for  procuring  what  is  needed. 


Table  13.  Examples  used  in  documents-  Competition. 


Example 

Reference 

Key  Points 

U.S.  Air  Force  PBL 

Programs 

Berkowitz  etal. 
(2003) 

OEM's  advantage:  The  Total  System  Support  Responsibility  (TSSR)  is  the  Air  Force's  approach  to  PBL.  Since  the  AF  weapon  systems 
(aircraft)  are  heavily  dependant  on  the  OEM,  it  is  a  natural  progression  to  continue  to  do  business  with  the  OEM  after  the  system  is 
deployed.  The  use  of  the  OEM  is  one  way  to  be  sure  that  the  expertise  exists. 

B-1B  Lancer 

Berkowitz  etal. 
(2003) 

Few  Suppliers:  According  to  authors;  one  of  the  lessons  learned  from  their  contract  experience  is  keeping  the  number  of  vendors 
small. 

Turkey  Navy: 
Transportation 

Bozkurt  and 
Guducu  (2005) 

Due  to  having  few  supplier  candidates,  authours  suggest  to  use  PBL  as  an  acquisition  strategy. 

A/C  Tires:  Michelin 

Dean (2002) 

PSI:  Michelin  is  the  PSI  of  this  program.  Michelin  just  supplies  the  items.  Othervendors  handle  all  other  process  such  as  stock 
control,  all  transactions,  demand  forecast,  etc. 

Japan  Maritime  Self- 
Defense  Force:  MCH-101 

Harada  (2010) 

Managing  Few  Suppliers:  The  PBL  contract  stakeholders  are  limited,  because  there  are  just  two  main  suppliers,  so  it  is  easier  for 
JMSDF  to  structure  its  organization  around  the  PBL  contract. 

U.S.  Navy  PBL  Programs 

Harada  (2010) 

Willing:  The  U.S.  Navy  prefers  to  use  PBLfor  expensive  or  delicate  items,  those  for  which  readiness  is  a  critical  issue,  and  hard  to 
manage  candidates.  It  also  says  that  "there  must  be  a  vendor  who  is  willing  to  contract  with  the  customer"  and  "the  vendor  must  be 
affordable  to  the  customer" 

12.  Properties  of  System 

In  seventeen  out  of  eighty  one  documents,  it  is  stated  that  properties  of  system  should 
affect  PBL.  All  literature  agrees  that  every  acquisition  does  not  fit  in  PBL.  For  instance  PBL  is 
mostly  good  for  new,  strategically  important,  high-tech,  complex,  long  life,  capital  intensive 
systems.  On  the  other  hand  PBL  is  not  good  at  all  for  legacy  systems  at  their  end  of  lives,  highly 
reliable  systems,  etc. 

Strategic  importance,  criticality  of  system  is  the  most  attractive  point  for  owners.  As 
Goure  (2011b,  p.2)  asks;  . .  what  is  more  important,  national  security  or  more  work  for  the 
ALC?”  And  regarding  the  cost;  which  is  important  reduction  in  costs  or  providing  national 
security?  President  Obama  stated:  “Our  defense  budget  isn't  about  politics,  it's  about  the  security 
of  our  country,  and  who  knows  that  every  dollar  wasted  is  a  dollar  we  can't  spend  to  care  for  our 
troops  or  protect  the  homeland.”  Therefore  one  of  the  most  important  points  is  to  be  ready  when 
a  critical  system  is  required. 
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According  to  Harada  (2010,  p.26)  “U.S.  Navy  prefers  to  use  PBL  for  expensive  or 
delicate  items,  those  for  which  readiness  is  a  critical  issue”,  and  for  “U.S.  Air  Force,  newer 
aircraft  tend  to  use  PBL  more  than  older  aircraft.”  After  stating  not  all  fits  in  PBL,  Harada  (2010, 
p.47)  lists  which  are  suitable  which  are  not:  “. . .  high-technology  weapon  systems  or  components 
are  good  candidates  for  PBL,  but  low-production  special  weapon  systems  or  highly  reliable 
weapon  systems  are  not  suitable  for  PBL.”  Buyukgural  (2009,  p.14)  agrees  Harada  (2010) 
regarding  high  technologies.  According  to  him,  “if  the  contractors  could  produce  a  highly 
technological  product  or  service,  then  they  should  know  how  to  improve  it  by  means  of 
reliability  and  cost  aspects.” 

In  addition  to  high-tech-systems;  Sols  and  Verma  (2007)  list  “large  scale,  long-life,  and 
capital  intensive  systems”  to  be  fit  in  PBL.  They  emphasize  the  intent  of  PBL;  improving 
perfonnance  while  reducing  logistics  footprints.  They  recommend  shifting  to  PBL  from 
traditional  approach  when  sufficient  life  time  has  been  left  for  the  system.  Sols  and  Verma 
(2007)  reports  this  shift  trend  in  their  study.  Kim  et  al.  (2006)  pointed  this  shift,  too;  but  this  time 
the  reason  is  the  complexity  of  systems.  Also  Geary  (2006,  p.75)  thinks  this  shift  is  “to  keep 
complex  systems  combat  ready.”  And  according  to  him  DoD  top  management  wants  to  use  PBL 
for  all  new  systems.  Gillie  (2006)  states  that  Boeing’s  intent  by  PBL  is  taking  complexity  out  of 
their  customers  operations. 

The  more  complex  the  systems  the  more  likely  owner  decides  to  buy  the  service.  “The 
increased  complexity  of  products  and  processes  is  compelling  more  and  more  companies  to  use 
structured  decision  tools,  to  facilitate  strategic  decisions  to  make  or  buy.”  (Bozkurt  and  Guducu, 
2005,  p.31)  Wharton  researchers  think  that  complex  systems’  owners  such  as  aerospace  and 
defense  need  more  sophisticated  acquisition  strategies  like  PBL  partnerships  instead  of 
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traditional  approaches  (Cohen  and  Netessine,  2007).  According  to  Berkowitz  et  al.  (2003); 
support  becomes  complex  moving  from  legacy  systems  to  new  systems.  Likewise  acquisition 
strategy  should  move  from  traditional  approach  to  future  strategies  such  as  PBL. 

On  the  other  hand  for  legacy  systems  close  to  their  end,  PBL  does  not  provide  any 
perfonnance  or  cost  improvement. 

“PBL  can  often  improve  reliability,  but  there  are  limitations,  particularly  on  legacy 
systems.  Long-standing,  systemic  reliability  problems  in  fielded  systems  are  unlikely  to 
be  corrected  without  appropriate  commitment  of  necessary  funding.  ...  In  some 
instances,  particularly  for  legacy  systems  approaching  retirement,  PBL  may  in  fact  not  be 
the  most  appropriate  support  solution.”  (Kobren,  2009,  p.260-261) 

But  PBL  should  be  used  for  legacy  systems  with  sufficient  life  to  decrease  costs.  Kobren 

(2010,  p.202)  recommends  using  PBL  to  decrease  increasing  sustainment  costs: 

“Supporting  and  sustaining  increasingly  complex,  often  aging  weapon  systems  in  an  era 
of  budgetary  austerity,  and  faced  with  a  variety  of  threats  and  challenges  from  both  state 
and  non-state  actors,  the  department  must  leverage  LCM  processes,  practices,  and 
policies,  coupled  with  perfonnance  based  life-cycle  product  support  sustainment 
strategies  to  preclude  degraded  readiness  and  upward  spiraling  support  costs.” 

Goure  (2010)  supports  this  idea  with  NAVICP  and  Navy’s  FIRST  examples.  According  to  him 

PBL  helped  Navy  to  deal  with  obsolescence  issue,  which  is  one  of  the  largest  issues  for  Navy. 

Both  programs  achieved  huge  successes. 

13.  Up-Front  Planning  and  Early  PBL  Implementation 

In  sixteen  out  of  eighty  one  documents  it  is  stated  that  early  decision  of  PBL  will  booster 

success.  PBL’s  goal  is  to  provide  the  desired  outcomes  such  as  readiness,  reliability.  It  is 

basically  an  acquisition  strategy  for  sustainment,  support.  Outcomes  are  linked  to  this  support 

program.  “The  link  between  performance  and  sustainment  is  critical,  and  must  be  considered 

throughout  the  early  program  design  activities.”  (DoD  Designing  and  Assessing  Supportability 

Guide,  2003,  p.8)  The  guide  also  mandates  PMs  starting  reliability,  availability,  and 
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maintainability  activities  early  in  the  acquisition  cycle.  All  plans  should  be  done  early  in  the  life 
cycle. 

Owner  wants  to  keep  the  system  ready.  Therefore  support  strategies  should  be  considered 
even  at  the  decision  making  progress  of  buying  the  system.  In  DoD  PMs  have  the  responsibility 
regarding  the  support  program.  “The  PM  shall  establish  logistics  support  concepts  (e.g.,  organic, 
two-  level,  three-  level,  contractor,  partnering,  etc.)  early  in  the  program  and  refine  the  concepts 
throughout  program  development.”  (DoDD  5000.02,  2002,  p.82) 

DoD  does  this  up-front  planning  well. 

“Including  sustainment  in  new  programs  means  conceptualizing  how  the  system  will  be 
sustained  as  it  is  being  designed,  how  the  modeling  capability  will  be  generated,  how  the 
system  will  be  serviced  in  the  field,  how  industry  and  service  branch  support  personnel 
will  partner  in  sustaining  that  asset,  and  how  clearly  define  of  the  multiple 
responsibilities.  Ideally,  the  sustainment  plan  is  developed  and  in  place  before  any 
acquisition.”  (Hedden,  2007,  p.2) 

Devries  (2004)  thinks  that  this  is  a  significant  shift  from  traditional  approach  to  an  early- 
in-the-life-cycle  planning.  PBL  includes  total  life  cycle  management  which  emphasizes  early 
focus  on  sustainment  in  the  program  management  office. 

As  mentioned  before  OEMs  have  advantages.  According  to  Berkowitz  et  al.  (2003),  one 
of  the  advantages  is,  OEM  may  interact  early  in  the  development/fielding  cycle  of  the  system. 
They  have  the  ability  to  play  role  in  the  early  stages.  In  that  case  it  would  be  an  important  role 
that  may  affect  performance  improvement  by  means  success  of  PBL.  It  may  even  decrease  costs. 
“Application  of  a  PBL  strategy  from  the  early  stages  of  product  design  and  development  can 
lead,  by  controlling  the  dominating  design  parameters,  to  significant  reductions  in  life-cycle 
costs.”  (Sols  and  Venna,  2007,  p.47)  DoD  Designing  and  Assessing  Supportability  Guide  (2003, 
p.8)  states  that: 
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“SOE  is  the  composite  of  performance,  availability,  process  efficiency,  and  total 
ownership  cost.  The  objectives  of  the  SOE  concept  can  best  be  achieved  through 
influencing  early  design  and  architecture  and  through  focusing  on  System  Design  for 
Operational  Effectiveness  (SDOE).  Reliability,  reduced  logistics  footprint,  and  reduced 
system  life  cycle  cost/total  cost  of  ownership  (TOC)  are  most  effectively  achieved 
through  inclusion  from  the  very  beginning  of  a  program  -  starting  with  the  definition  of 
required  capabilities.” 

Five  documents  used  examples  to  express  their  support  to  the  ideas  above.  Table  14 
shows  the  examples  used  by  corresponding  documents. 


Table  14.  Examples  used  in  documents-  Up-front  Planning,  Implementation. 


Example 

Reference 

Key  Points 

JSF 

Blum  berg  (2006b) 

The  program  was  a  PBL  program  from  the  start.  Prior  to  the  start  of  system  design  and  development  (SDD)  in  fall  2001,  fully  validated 
and  affordable  operational  requirements  were  available.  Vitasek  and  Kotlanger  agree  that  the  JSF  program  is  on  target  with 
sufficient  long-range,  up-front  planning  to  provide  a  high  probability  of  success. 

U.S.  Army/AAI 

Unmanned  A/Cs 

Blum  berg  (2007a) 

The  Army's  acquisition  strategy  was  to  buy  an  off-the-shelf  system.  It  is  a  fly-before-you-buy  competition,  so  the  product  design 
was  complete  as  part  of  the  proposal  process.  The  Army  wanted  to  field  it  quickly,  learn  from  soldier  experience,  then  dial  it  in  and 
modify  it  for  more  military  utility. 

U.S.  Air  Force  Traditional 
Acquisition  Strategy : 

Test  systems  in  legacy 
aircsrafts 

Mitchell  (2008) 

This  is  a  good  example  of  a  system  that  wasn't  particularly  well  managed  in  the  first  place.  But  the  problem  isn't  just  a  matter  of 
poor  obsolesence  planning  on  the  customer's  part.  It's  also  a  direct  consequence  of  a  business  model  that  promotes  obsolesence  by 
paying  the  same  vendors  who  build  the  systems  to  maintain  them  afterwards.  These  vendors  have  a  financial  disincentive  to  design 
systems  that  last  longer  because  they'll  make  less  revenue  on  the  back  end.  It  wasn't  the  fault  of  the  people  who  made  the  system 
at  the  beginning.  The  customer  didn't  require  it  to  be  managed. 

Army  -  ARC- 190 

Pettingill  et  al 
(2004) 

The  ARC-190's  PBL  planning  team  was  identified  as  a  best  practice  for  their  up-front  and  early  planning.  The  ARC- 190  program  is 
looking  at  PBL  as  a  means  for  converting  this  organically  repaired  item  into  a  contractor  repaired  item. 

Generic  Example: 

Future  Combat  Vehicle 

Trovato (2004) 

Minimum  maintanance  and  onboard  support  is  integrated  with  design.  Early  emphasis  on  design  will  provide  reduction  in  costs  and 
improved  performance. 

As  seen  design  is  the  first  piece  of  up-front  planning.  One  of  the  courses  is  given  by  DAU 
is  about  design  for  sustainment: 

“This  3 -day  course  emphasizes  the  application  of  engineering  and  related  technical 
methodologies  to  optimize  the  design  of  systems  for  supportability,  not  only  at  initial 
fielding  but  throughout  the  service  life.”  (Cothran,  2008,  p.36) 

Up-front  planning  and  early  PBL  implementation  in  the  life  cycle  is  a  requirement 

according  to  Heron  (2010).  It  is  the  best  way  to  reduce  life  cycle  costs  and  improve  perfonnance. 

Buyers  should  pay  attention  to  that,  before  making  the  decision  to  buy  a  new  system. 

14.  Other  Important  Points:  Risk  and  Stage  of  PBL 

As  mentioned  at  the  beginning  of  this  section;  risk  and  stage  of  PBL  are  frequently 

repeated  in  literature.  Risk  is  mentioned  in  twenty  eight  documents  and  stage  of  PBL  is 

mentioned  fourteen  documents  over  eighty  one.  At  first  sight  they  seem  to  be  affecting  the 
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success  of  PBL,  but  when  it  is  examined  in  details,  it  can  be  easily  stated  that  they  are  not  factors 
affecting  success.  Actually  they  are  affecting  success  via  the  other  factors  mentioned  earlier. 

14.1  Risk 

Risk  is  included  in  every  type  of  acquisition.  In  PBL  it  is  transferred  to  PSI.  PSI  absorbs 
this  risk  by  increasing  contract  value,  length  or  type.  Harada  (2010,  p.21-24)  lists  kinds  of  risks 
regarding  PBL.  These  risks  may  be  linked  to  a  factor  mentioned  in  this  study.  ‘Supply  chain 
risks’  will  be  shared  by  the  partners  of  PBL;  ‘skill  drain’  will  be  handled  by  PBL  teams; 
‘assessing’,  ‘cost’,  ‘price’  and  ‘evaluation’  are  all  matter  of  data  requirement.  Contractor  will 
negotiate  all  with  buyer  and  a  fair  contract  will  be  initiated.  Issues  regarding  data  requirement  is 
also  pointed  by  Newsome  (2008). 

“Since  the  contractors  are  responsible  for  achieving  the  perfonnance  requirements  placed 
in  the  statements  of  work,  the  majority  of  the  risk  related  with  perfonnance  is  transferred 
from  government  to  industry.  Agencies  should  consider  this  when  determining  the 
appropriate  acquisition  incentives.”  (Bozkurt  and  Guducu,  2005,  p.20) 

In  PBL  operational  risks  are  shifted  to  contractor.  To  handle  this  risk  contractor  should 

invest  and  contractor  wants  to  gain  profit  with  this  investment. 

“PBL  contracts  are  almost  always  offered  across  multiple  years  (lowering  financial  risk 
for  the  vendor),  with  the  expectation  that  the  vendor  will  assume  some  degree  of 
operational  risk.  Figure  3  shows  the  expected  assumption  of  operational  risk  by  the 
vendor.”  (Doerr  et.  al,  2005,  p.170) 

Cohen  and  Netessine  (2007)  points  out  the  uncertainties  especially  in  complex  systems. 
According  to  them  this  type  of  risk  can  be  averted  by  contract  type;  using  cost  plus  contracts 
instead  of  fixed  price.  Blumberg  (2007a)  states  so;  contracting  negotiations  will  take  care  of  risks 
involved.  “When  properly  designed,  a  PBL  contract  will  actually  diminish  the  risks  borne  by 
both  partners.”  (Vitasek  and  Geary,  2007,  p.  1) 
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Gardner’s  (2008)  study  shows  that  PBL  users  believe  that  all  types  of  risks  such  as  lack 

of  data,  financial,  return  of  investment,  rapid  change  of  costs,  etc.  can  be  altered  by  incentives 

such  as  contract  length,  award  terms,  and  type  of  contract.  According  to  him  buyer  and 

contractor  should  negotiate  and  share  the  risks.  Kobren  (2009)  agrees  regarding  the  incentives. 

According  to  Sols  and  Verma  (2007);  risk  will  be  absorbed  through  contractual  framework.  Kim 

et  al.  (2006)  tried  to  find  out  the  best  contract  to  handle  risks.  Their  formulation  includes  the 

factors  that  affects  success  and  link  them  to  risk  factor. 

Another  absorber  will  be  competition.  PMs  should  use  an  open  systems  approach  to 

achieve  mitigating  “the  risks  associated  with  technology  obsolescence,  being  locked  into 

proprietary  technology,  and  reliance  on  a  single  source  of  supply.”  (DoDD  5000.02,  2002,  p.83) 

DoDD  5000.02  (2002)  mandates  risk  management  and  states  that  using  best  practices, 

best  capabilities  will  reduce  the  risk.  Therefore  applying  best  practices  is  another  absorber. 

Lessons  Learned  (2004)  shows  that  partnership,  strategic  alliance  has  reduced  the 

program  risks  especially  for  JSTARS.  According  to  Lessons  Learned  (2004)  sharing  risk  with 

partnership  is  another  way  to  handle  risks;  Navy  Aircrew  Survival  Equipment  had  this 

experience.  Vitasek  et  al.  (2007)  agree;  partnership,  sharing  risk  are  helpers  to  reduce  risks. 

According  to  Vitasek  and  Geary  (2008)  flexibility  will  balance  the  risks  involved. 

As  a  result  risk  is  affecting  success  of  PBL  but  it  affects  through  the  other  factors. 

14.2  Stage  of  PBL 

“There  are  four  levels  for  PBL: 

Level  1 :  components  such  as  aircraft  tires, 

Level  2:  major  subsystems  such  as  aircraft  engines, 

Level  3:  platfonn  availability  such  as  F-l  17  Nighthawk,  and 

Level  4:  Mission,  setting  the  stage  for  the  future — achieving  true  pay  for  performance 
such  as  the  Army  shadow  tactical  unmanned  aerial  vehicle  program.”  (Harada,  2010, 
P-25) 
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Vitasek  et  al.  (2006a),  Kratz  and  Buckingham  (2010a)  define  four  levels  likely  and  give 
similar  examples  in  details.  Owners  should  determine  which  level  is  appropriate  for  them.  Doerr 
et.  al  (2005)  thinks  this  categorization  as  a  level  of  commercial  involvement. 

Gardner’s  (2008)  study  shows  that  stage  of  PBL  will  affect  negotiations  a  little  regarding 
contract  length,  award  terms,  and  type  of  contract.  Similarly  awarded  PBL’s  have  a  long  range 
from  level  1  to  level  3.  (There  is  not  any  level  4  PBL  yet.)  “The  DoD  has  credited  PBL  with  a 
long  string  of  successes,  ranging  from  logistics  support  for  simple  parts  such  as  tires  to 
subsystems  such  as  engines  —  and  in  some  cases,  to  full  weapon  systems.”  (Newsome,  2008, 
p.5;  Vitasek  and  Geary,  2007,  p.l) 

On  the  other  hand  stage  of  PBL  will  affect  the  implementation  processes,  partnership, 
and  alliances.  Cothran  (n.d.)  points  this  out  but  he  does  not  conclude  any  effect  on  success  of 
PBL.  “Performance  based  arrangements  are  successful  at  the  component,  subsystem,  and  system 
level,  depending  upon  the  unique  circumstance  of  the  system.”  (Kratz  and  Buckingham,  2010a, 
P-61) 

Consequently  stage  of  PBL  doesn’t  have  any  effect  on  success.  It  may  affect  type, 
properties  of  contacts.  DoD  achieved  numerous  successes  in  every  stage. 
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III.  Methodology 


Overview 

In  this  section  the  tool  for  confonnity  assessment  of  PBL  is  built  according  to  the 
literature  review  which  provided  for  thirteen  factors  that  have  an  effect  on  success  of  PBL.  How 
these  factors  are  affecting  success  is  explained  in  literature  review,  too.  Therefore,  qualitative 
infonnation  is  in  hand  so  far. 


This  tool  assumes  a  linear  relationship  between  the  success  of  PBL  and  each  factor.  To 
build  the  tool,  qualitative  information,  ideas  from  the  literature  are  converted  to  quantitative 
scales  to  evaluate  each  factor.  To  get  a  score  from  these  scales,  the  user  should  answer  the 
questions  asked  by  the  tool  regarding  each  factor;  therefore,  a  more  objective  evaluation  may  be 
obtained.  The  tool  also  lets  the  user  make  his/her  own  evaluation;  a  subjective  one.  In  addition  to 
those  evaluations,  each  factor  is  given  a  weight  corresponding  to  their  occurrence  frequency  in 
the  sample  literature  and  also  another  weight  for  their  importance  as  explained  in  the  literature. 
The  tool  also  lets  the  user  detennine  a  customized  weight.  The  basic  formulation  is  below: 

13 

r,  .  V  i  fx„  score  from  evaluation  of  nth  factor 

fix)  =  >  wn.xn  where]  .  , 

Z_i  (.  wn  is  the  corresponding  weight 

n= 1 

Scores  range  from  0  to  10  and  the  sum  of  all  weights  is  equal  to  1.  The  general  formula 
for  occurrences’  frequency  weight  is: 

#  of  occurances  in  the  literature  sample 


Wr,  = 


T#  of  all  factors'  occurances  in  the  literature  sample 
As  a  result;  the  tool  concludes  6  (=  2  scores  x  3  weights)  calculations.  These  are: 


f{x)  =  user  determined  evaluation  x  user  determined  weight  (wuser  n) 
fix)  —  user  determined  evaluation  x  occurances'  weightiwoccurancesn) 
fix)  —  user  determined  evaluation  x  author  determined  weightiwauthor  n) 
fix)  —  tool  evaluation  x  user  determined  weight  iwusern) 
fix)  —  tool  evaluation  x  occurances'  weight  iwoccurancesn) 
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/(x)  =  tool  evaluation  x  author  determined  weight  (wauthor  n) 

The  tool  is  pictured  at  Appendix  B.  The  following  subsections  are  describing  each 
factors’  scales,  how  to  assess  them  and  their  weight  calculation.  And  the  last  subsection 
describes  general  score  calculation  and  other  specifications  of  the  tool. 

1.  Reduction  in  Total  Cost  of  Ownership,  Life  Cycle  Cost 

Cost  avoidance  is  significant  for  many  of  the  PBL  programs.  TOW-ITAS  PBL  program’s 
cost  avoidance  ranges  from  25.82%  to  60.58%  per  year  from  2001  to  2005;  approximately  33% 
over  7  years.  In  GE  F404,  Navy  experienced  13.8%  total  cost  of  ownership  reduction.  Shadow 
UAV  PBL  program  achieved  28%  contract  cost  reduction;  F-22  program  achieved  40  %, 

Tornado  achieved  30%;  JAVELIN  is  estimated  to  achieve  10%  and  E-3D  AW  ACS  is  estimated 
to  achieve  12%  total  cost  reduction. 

Another  point  that  may  affect  total  cost  for  organization  may  be  reducing  human 
resources  requirement  by  PBL  teaming.  F-l  17  PBL  program  is  an  example  for  that.  18. 1 1% 
reduction  in  employee  requirement  was  achieved  in  addition  to  maintenance  cost  reduction. 

Another  savings  has  been  experienced  in  inventory  reduction.  APU  PBL  program  is  the 
best  example  to  understand  it.  While  $35  million  savings  is  achieved  in  total  costs  savings,  $8.5 
million  cost  avoidance  is  occurred  in  inventory  annually.  That  means  24.29%  of  total  costs 
savings  is  generated  by  inventory  reduction.  In  GE  F404  Navy  experienced  21%  inventory 
reduction.  In  NAVICP  A/C  tire  PBL  program  the  inventory  reduction  is  huge;  75%.  By  Tornado 
radar  support  PBL  program  inventory  went  down  to  50%. 

As  a  result;  the  minimum  expected  total  cost  reduction  is  10%  and  the  maximum 
achieved  is  40%  over  programs’  length.  Evaluation  scale  is  proportional  to  this  range.  Zero 
percentage;  any  change  in  total  cost  of  ownership  will  get  0  and  10  goes  to  expectation  of  40  or 


89 


more  %  reduction  in  total  cost  of  ownership.  In  most  cases,  military  do  not  take  human  resource 
requirement  or  inventory  into  account  while  calculating  total  costs.  Therefore  tool  offers  user  to 
evaluate  these  sub-items,  in  exchange  of  first  question.  Selecting  any  percentage  except  not 
applicable  (N/A)  option,  will  direct  user  to  use  only  total  cost  reduction  in  assessment 
calculations.  Otherwise  it  is  offered  to  fill  ‘contract  cost  reduction  estimation’,  ‘human  resource 
reduction  expectation’  and  ‘inventory  reduction  expectation’.  ‘Contract  cost  reduction 
estimation’  is  scaled  just  like  ‘total  cost  reduction  estimation’;  0%,  there  is  not  any  change  in 
total  cost  of  ownership  will  get  0  and  10  goes  to  expectation  of  40  or  more  %  reduction  in  total 
cost  of  ownership.  And  this  sub-item  creates  the  50%  of  this  factor’s  score.  For  ‘human  resource 
reduction’  scale,  F-l  17  PBL  program  is  taken  as  reference.  18%  or  more  reduction  will  get  10 
and  other  entries  lower  than  18%  will  get  proportional  values.  For  inventory  25%  and  more 
reduction  will  get  10  and  other  entries  lower  than  25%  will  get  proportional  values.  These  two 
sub-items  creates  the  other  half  of  this  factor’s  score. 

Cost  reduction  is  one  of  the  most  reoccurring  factors.  Its  occurrences’  weight  will  be  the 
highest. 

#  of  occurances  in  the  literature  sample  60 

^occurance,  1  77~7  7-  TTT  I  >  ToT  12.47% 

Z#  of  all  factors  occurances  481 

Although  most  of  the  documents  count  this  factor  as  one  of  the  purposes  of  PBL,  DoD 
defends  itself  against  GAO  reports  by  stating  PBL’s  aim  is  not  reducing  costs,  it  is  improving 
performance,  reliability,  readiness.  In  addition  to  that  as  stated  in  literature  review  section, 
authorities  find  national  security,  soldiers’  safety  more  important  than  costs,  budget  issues. 
Therefore  this  factor’s  weight  should  be  lower  than  the  occurrences’  weight.  The  weights  of  all 
factors  range  from  3%  to  12%.  7.69%  is  the  equally  shared  weight  among  13  factors.  5%  is 
determined  to  have  a  fair  weight  for  this  factor. 
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w author  ,1 


As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

wUSer, i  —  wi/Z  be  determined  by  user 
2.  Presence,  Determination  of  Metrics,  Performance  Outcomes 

Perfonnance  outcomes,  metrics  are  the  key  components  of  PBL.  PBL  is  an  acquisition 
method  which  buys  performance.  Therefore  first  rule  is  identifying  the  perfonnance  requirement 
accurately.  Metrics,  outcomes  should  be  aligned  to  goals  of  the  systems,  organization.  Metrics 
should  be  understandable,  clear,  specific,  traceable  and  measurable  such  as  in  terms  of  quality, 
timeliness,  quantity,  etc.  Metrics  should  be  negotiated  by  all  partners  to  be  fair  to  all. 

Subject  matter  experts  suggest  to  use  5  top  level  metrics;  operational  availability, 
operational  reliability,  cost  per  unit  usage,  logistics  footprint  and  logistics  response  time.  Metrics 
should  be  related  to  them  at  least.  In  addition  to  them;  customer  satisfaction  is  mentioned  as  a 
metrics  by  a  couple  of  authors. 

Buyer  tries  to  buy  outcomes;  therefore  s/he  should  have  a  baseline  and  a  target  to  reach. 
Having  a  baseline  is  related  to  historical  data.  But  a  target  value  should  be  set  for  metrics. 

Early  implementation  of  metrics  is  another  suggestion  by  some  authors  regarding 
detennination  of  metrics,  but  this  is  an  issue  regarding  ‘up-front,  early  planning’  factor. 

Subject  matter  experts  also  suggest  using  5  or  less  metrics  to  focus  on  what  is  required, 
not  to  lose  insight  of  main  goal. 

According  to  the  information  above  summarized  from  literature  review;  1 1  key  questions 
are  gathered  to  assess  ‘metrics  determination’  factor.  Table  15  lists  these  questions. 
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Table  15.  Questions  for  Presence,  Determination  of  Metrics. 


No 

Question 

1 

How  accurately  are  the  requirements  of  goods/service  defined? 

2 

How  well  are  the  metrics  aligned  to  organization's  goals? 

3 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

4 

How  realistic,  sound  are  the  metrics? 

5 

How  traceable  are  the  metrics? 

6 

How  measurable  are  the  metrics? 

7 

Are  the  metrics  negotiated  with  candidates? 

8 

How  lair  are  the  metrics  lor  all  partners? 

9 

Are  metrics  expressed  in  any  of  five  top  level  metrics? 

10 

Is  a  target  value  set  for  each  metrics? 

11 

How  many  metrics  are  planned  to  use  in  contract? 

Each  question  has  similar  importance  in  evaluation.  Therefore  they  are  weighted  equally. 
Each  question  has  a  score  range  of  0-10,  so  do  Yes/No  questions.  0  means  not  at  all  and  10 
means  extremely.  For  1 1th;  5  or  less  will  get  10,  more  will  get  0.  The  score  for  the  factor  is  the 
mean  of  all  scores. 

Metrics  determination  is  one  of  the  most  reoccurring  factors.  Its  occurrences’  weight  will 
be  high. 

#  of  occurances  in  the  literature  sample  56 
occurance, 2  y#  0f  all  factors'  occurances  481 

Metrics  is  the  key  of  PBL;  therefore  its  weight  should  be  in  accordance  with  its 
importance.  The  weights  of  all  factors  range  from  3%  to  12%.  7.69%  is  the  equally  shared 
weight  among  13  factors.  12.5%  is  determined  to  have  a  fair  weight  for  this  factor: 

w author, 2  ~  12.5% 

As  mentioned  before  the  user  may  give  his/her  customized  weight  to  this  factor: 
wuser, 2  =  w til  he  determined  by  user 
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3.  Performance  Improvement 

The  very  most  intent  of  PBL  is  improving  desired  metrics.  That  is  not  only  what  DoD 
states,  but  also  what  subject  matter  experts  admit.  Literature  has  several  examples  for  this  factor. 
To  create  a  scale  some  of  them  are  examined  in  this  section,  too.  There  is  also  some  emphasis  on 
designing,  redesigning  and  suggests  on  early  implementation  for  improving  metrics;  but  these 
are  subjects  of ‘up-front,  early  planning’  factor. 

In  GE  F404,  Navy  experienced  46%  fleet  availability  improvement,  49%  component 
availability  improvement.  APU  PBL  program  has  similar  achievements;  around  30%  availability 
improvement,  15%  reliability  improvement  (30-60%  reliability  improvement  is  guaranteed  in 
contract).  For  F/A-18E/F  FIRST  Program;  16%  MICAP  rate  improvement,  18%  aircraft 
availability  improvement  are  announced.  A/C  Tires  PBL  program  increased  availability  by  17%, 
delivery  time  improvement  by  10%.  In  F-l  17  program;  time  on  wing  increased  by  85%,  response 
time  decreased  by  71%,  and  turn  times  decreased  by  25%  for  engine,  43%  for  LRUs.  FLIR 
experienced  40%  improvement  in  system  reliability,  and  14%  availability  improvement.  Table 
16  summarizes  these  examples  to  provide  a  better  view  for  scale  range. 

Table  16.  Performance  Improvements  for  successful  PBL  programs. 


Improvement  \  PBL  Program 

GEF404 

APU 

FIRST 

F-22 

A/C  Tires 

F-117 

FLIR 

Top  Level  Metrics  Improvement 

46% 

30% 

16% 

15% 

10% 

85% 

40% 

Top  Level  Metrics  Improvement 

49% 

15% 

18% 

15% 

17% 

71% 

14% 

Top  Level  Metrics  Improvement 

- 

30-60% 

- 

68% 

- 

25-43% 

- 

Although  the  average  percentage  improvement  is  35%  according  to  the  table;  A/C  tires 
PBL  program  is  counted  as  successful  with  a  10%  and  17  %  improvement.  Therefore  for  this 
factor  10  point  goes  to  20%  and  more  improvements;  lower  improvement  ratios  will  get 
proportional  points  for  each  metrics  used  in  contract.  As  mentioned  before;  literature  advises  to 
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use  5  or  less  metrics.  Therefore  the  first  5  metrics  is  scored  individually  and  for  others  user 
should  give  an  average  improvement  ratio. 

This  factor  is  mentioned  a  lot  in  the  literature,  too.  Therefore  occurrences’  weight  will  be 
correspondingly  high. 

#  of  occurances  in  the  literature  sample  52 
woccurance, 3  Z#  of  all  factors' occurances  481  10.81 /o 

PBL’s  one  of  the  intent  is  increasing  perfonnance  metrics.  Therefore  although 
occurrences’  weight  is  correspondingly  high,  it  should  be  weighted  higher,  as  high  as  ‘metrics 
determination’. 

w author, 3  ~  12,5% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

w user, 3  —  wiZZ  be  determined  by  user 

4.  Partnership,  Strategic  Alliance 

PBL  is  a  partnership,  an  alliance.  PBL  requires  more  sophisticated,  complex  relationships 
than  traditional  methods.  It’s  not  a  business  relationship,  not  a  traditional  arm’s  length 
relationship  anymore.  According  to  the  literature;  a  PBL  environment  should  at  least  have  long 
tenn  relationships,  and  strategic  alliances  have  the  best  practices.  Examples  of  successful  PBL 
partnerships  are  several.  According  to  these  examples;  partnerships  include  integration  with 
partners,  trust  among  partners,  knowledge  sharing,  and  even  physical  assets/network  sharing. 
Sharing  risks  and  benefits  is  also  essential.  It  requires  a  closed  managed  relationship.  This  is  a 
culture  change,  therefore  not  only  management’s  support  but  also  employees’  contribution  is 
required. 
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Tool  has  two  ways  to  evaluate  this  factor.  One  of  them  is  just  selecting  the  expected  level 
of  relationship  for  that  specific  acquisition.  It  includes  6  different  options  from  ‘Traditional 
Ann’s  Length  Relationships’  to  ‘Strategic  Alliance’.  Table  17  shows  the  corresponding  scores. 

PBL  requires  at  least  long  term  relationships 
and  as  stated  above  strategic  alliances  have  the 
best  practices.  Arm’s  length  relationships  do  not 
have  any  contribution  to  PBL.  Other  4  types 
have  corresponding  scores  according  to  this 
information. 

Traditional  arm’s  length  is  the  oldest  way  of  supplier-buyer  relationship,  where  you  do 
not  pay  attention  to  manage  or  improve  relations.  Close-Short  term  relationship  represents  a 
better  relationship  than  traditional  approach,  some  information  may  be  shared,  and 
improvements  might  be  seen  in  relations.  Long  term  relationship  represents  where 
communication  processes  got  more  formal,  commitment  might  be  seen  between  supplier  and 
buyer.  Limited  partnership  is  a  kind  of  partnership  but  in  this  case  all  of  the  resources  are  not 
combined  to  reach  the  mutual  goal.  Partnership  is  an  agreement  that  buyer  and  supplier  commit 
themselves  to  reach  mutual  goals,  benefits.  Strategic  alliance  is  the  expanded  version  of 
partnership;  partners  set  their  vision,  strategy  through  the  mutual  objectives,  this  is  synergy. 

The  other  way,  offered  by  tool,  is  to  evaluate  this  factor  via  assessing  sub  items  of 
partnership.  Each  item  has  a  score  range  of  0-10  and  has  equal  weight  for  evaluation  result.  0 
means  not  at  all  and  10  means  extremely.  Sub  items  are  listed  in  table  18.  User  should  answer 
all. 


Table  17.  Scores  for  Types  of  Relationships. 


Relationship 

Score 

Traditional  Arm's  Length 

0 

Close  -  Short  Term  Relationship 

2 

Long  Term  Relationship 

5 

Limited  Partnership 

7 

Partnership 

8 

Strategic  Alliance 

10 
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Table  18.  Questions  for  Relationships  Evaluation. 


No 

Question 

1 

What  type  of  relationship  is  expected  to  be  built  with  provider? 

2 

How  much  integration  is  expected  to  be  built  with  provider? 

3 

How  trustful  are  the  partners  to  each  other? 

4 

How  much  information  is  planned  to  be  shared  with  partners? 

5 

How  much  of  physical  network  is  planned  to  be  shared  among  partners? 

6 

How  much  risk  will  be  shared  among  partners? 

7 

How  much  of  benefits  will  be  shared  among  partners? 

8 

How  well  the  relationships  are  managed? 

9 

How  supportive  is  the  management  to  culture  change? 

10 

How  accceptable  is  this  culture  change  among  employees? 

This  factor  is  mentioned  a  lot  in  the  literature,  too.  Therefore  occurrences’  weight  will  be 
correspondingly  high. 


w, 


#  of  occurances  in  the  literature  sample  49 


occurance,  4 


=  10.19% 


T#  of  all  factors’  occurances  481 

Partnership  is  a  key  for  PBL.  Therefore  it  is  so  important.  But  the  literature  gives  it  an 
adequate  emphasis.  As  a  result  a  similar  weight  is  given  by  author. 


w  author,  4  10% 

As  mentioned  before  user  may  give  his/her  own  weight  to  this  factor: 

wUSer, 4  —  wZZZ  be  determined  by  user 


5.  Length  of  Contract 

PBL  requires  long  tenn  relationships  with  partners;  therefore  the  contract  length  should 
be  long  enough  to  provide  this  kind  of  relationship.  Subject  matter  experts  agree  that  the  most 
incentive  factor  is  contract  length  for  balancing  risk  and  up-front  investment.  And  also  buyer’s 
intent  is  having  long  term  support  by  PBL.  It  is  also  agreed  that  shorter  term  contracts  limit  PBL 
at  all.  So  the  question  is  ‘How  long  contract  length  should  be  for  a  successful  PBL?’ 


96 


Detailed  studies  show  that  5  years  base  contracts  with  award/option  terms  will  satisfy  all 
partners  with  return  on  investment  and  desired  outcomes.  A  five  years  base  contract  is  agreed  by 
both  organic  and  commercial  representatives  to  get  benefits.  Literature  is  full  of  benefits  of 
multi-year  contracts  as  mentioned  in  previous  sections.  On  the  other  hand  it  is  also  determined 
that  a  ‘less  than  3  years  contract’  is  not  beneficial  for  either  of  partners.  HIMARS  is  the  only 
successful  example  for  3  years  contracted  PBL. 

Providers’  will  is  to  get  returns  on  investment  and  cash  flow  continuity,  while  buyer 
would  like  to  have  a  sufficient  performance  level  of  desired  outcome. 

Opposers  state  that  very  long  tenn  contracts  limit  the  flexibility  of  buyer,  competition  in 
market.  Therefore  extremely  long  contracts  are  not  well  enough  either,  although  they  had  success 
as  in  Survey  King,  and  Merlin  programs  in  U.K.  Besides  competition  is  an  issue  regarding 
another  factor;  ‘candidates  and  competition’. 

Consequently;  less  than  3  years  contract  will  get  0;  3  to  5  years  will  get  4  and  5  points 
respectively;  5  to  10  will  get  ‘5+proportional  points’  up  to  10  points;  10  years  and  more  will  get 
10  points.  If  the  contract’s  length  in  years  is  not  seemed  to  be  an  appropriate  way  to  assess  this 
factor,  user  may  select  to  evaluate  benefits  to  get  a  score.  All  benefits  have  a  score  range  of  0-10 
and  have  equal  weight  for  evaluation  result.  0  means  not  at  all  and  10  means  extremely.  The  sub 
items  should  be  answered  are;  return  on  investment  allowance,  cash  flow  continuity  for  provider, 
and  the  sufficiency  of  the  contract  length  to  accomplish  a  certain  perfonnance  level  of  the 
desired  outcomes. 

Contract  length  is  mentioned  a  lot  in  the  literature  sample  as  an  effective  key  success 
driver.  Its  occurrences’  weight  will  be  correspondingly  high. 

#  of  occurances  in  the  literature  sample  41 

^ occurance, 5  7  } ,  r  7  i  T7TT  8.52% 

’  Z#  of  all  factors  occurances  481 
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A  similar  weight  is  given  by  author. 


w author,  5  9% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

w user, 5  =  w HI  be  determined  by  user 

6.  Flexibility 

As  mentioned  before;  PBL’s  purpose  is  buying  outcomes,  not  dictating  what  to  do.  In 
PBL  environment  it  is  advised  to  tell  the  contractor  what  is  needed,  not  how  to  achieve  it.  As 
long  as  the  buyer  gets  what  s/he  wants,  contractor  has  the  flexibility  how  the  support  is  provided. 
PBL  allows  partners  to  use  their  knowledge,  innovation,  experience.  By  PBL  responsibilities 
and  authorization  is  decentralized  to  innovate  and  apply  their  skills.  Providers  should  be 
empowered.  There  is  a  necessity  of  freedom  for  contractor’s  actions  to  do  what  s/he  does  best. 
Freedom  is  also  required  in  investment  for  contractor  against  ambiguity  and  risk.  This  is  a 
chance  for  contractor  to  decide  how  to  provide  the  requirements.  Buyer  wins,  too;  s/he  will  get 
what  is  desired. 

Besides  to  use  PBL  with  full  capacity  not  only  ‘how  to  do’  specifications  should  be 
eliminated  but  also  financial,  political,  statutory  barriers  should  be  taken  care  of. 


According  to  the  information  above,  tool  asks  user  the  questions  shown  in  table  19. 
Table  19.  Questions  for  Flexibility. 


No 

Questions 

1 

How  much  freedom  is  provided  for  contractor's  actions? 

2 

How  much  decentralized  is  the  authority? 

3 

How  much  decentralized  are  the  responsibilities? 

4 

How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what1  and  'how  to  do's? 

5 

How  constraining  are  financial,  political,  statutory  barriers? 
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Each  has  0-10  score  range.  0  means  not  at  all  and  10  means  extremely.  The  evaluation 
score  will  be  average  of  the  five  questions’  answers.  5th  and  6th  questions  are  barriers  for 
flexibility;  therefore  their  score  will  influence  overall  score  negatively.  They  are  converted 
automatically  by  the  tool. 

Flexibility’s  weight  of  occurrences  will  be  in  the  middles. 

#  of  occurances  in  the  literature  sample  39 

W occur ance, 6  7777  7  j ,  ,  7  i  TrTT  8.11% 

’  Z#  of  all  factors  occurances  481 

This  is  an  important  factor  for  PBL.  A  slightly  higher  weight  is  given  by  author. 

w author ,6  ~  10% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

w user, e,  —  wZZZ  be  determined  by  user 
7.  Better  Service,  Best  Value  Creation,  Best  Practices 

PBL  is  integration  of  partners  to  find  the  way  to  reach  best  practice.  It  is  the  combination 
of  all  organic  and  commercial  abilities.  As  mentioned  before;  abilities  include  managing  life 
cycle  systems  engineering  influence,  directing  supply  chain  management  decisions,  and  trying  to 
meet  the  operational  sustainment  metrics.  In  addition  to  skills;  knowledge  and  experience 
regarding  the  system  are  important.  The  other  important  factor  that  some  authors  emphasized  is 
quality  of  manpower.  It  is  also  important  to  find  the  best  mix  of  qualified  personnel.  For  organic 
structure  skilled  personnel  is  required  not  only  for  the  job  to  be  done,  but  also  for  acquisition 
processes. 

Cooperation  of  organic  and  commercial  elements  should  be  aligned.  Work  allocation 
should  be  done  according  to  who  does  the  best  job. 
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Depending  on  this  information  14  questions  are  asked  to  the  user.  Table  20  lists  the 


questions. 


Table  20.  Questions  for  Better  Service,  Best  Value  Created,  Best  Practices. 


No 

Question 

1 

How  well  will  the  expected  service  quality  be  provided  by  organic  structure  in  this  partnership? 

2 

How  well  will  the  expected  service  quality  bw  provided  by  commercial  structure  in  this  partnership? 

3 

How  capable  is  the  organic  structure  to  manage  life  cycle  of  systems? 

4 

How  qualified  is  the  organic  structure  for  directing  supply  chain? 

5 

How  knowledgable  is  the  organic  structure  about  system? 

6 

How  experienced  is  the  organic  structure  about  system? 

7 

How  skilled  is  the  organic  manpower  regarding  job  to  be  done? 

8 

How  skilled  is  the  acquisition  personnel  regarding  job  to  be  done? 

9 

How  capable  is  the  commercial  structure  to  manage  life  cycle  of  systems? 

10 

How  qualified  is  the  commercial  structure  for  directing  supply  chain? 

11 

How  knowledgable  is  the  commercial  structure  about  system? 

12 

How  experienced  is  the  commercial  structure  about  system? 

13 

How  skilled  is  the  commercial  manpower  regarding  job  to  be  done? 

14 

How  well  is  the  work  allocation  done  between  partners  according  to  best  practices? 

Each  has  0-10  score  range.  0  means  not  at  all  and  10  means  extremely.  The  first  two 
questions  ask  the  service  quality  of  both  organic  and  commercial  structure  and  directly  evaluate 
the  factor  by  taking  average  of  these  two.  For  commercial  structure  user  should  decide  the  level 
among  the  candidates  according  to  their  probability  of  being  the  partner.  S/he  may  decide  to  use 
the  most  possible  candidate’s  score,  weighted  score  or  average  of  all.  If  user  prefers  to  evaluate 
by  sub  questions,  the  evaluation  score  will  be  average  of  these  answers.  And  again  for 
commercial  structure  user  should  decide  the  score  among  the  candidates  according  to  their 
probability  of  being  the  partner. 

This  factor’s  weight  of  occurrences  will  be  in  the  middles,  too. 

#  of  occurances  in  the  literature  sample  38 
w°ccurance,7  Z#  of  all  factors’  occurances  481 

Literature  gives  adequate  emphasis  on  this  factor.  Therefore  a  similar  weight  is  given  by 

author. 


100 


w author, 7  ~  8% 


As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

w user, 7  —  wiZZ  be  determined  by  user 
8.  PBL  Experience,  Awareness  and  Training 

PBL  is  a  new  culture,  a  transition  from  traditional  approaches.  The  organizations  should 
deal  with  implementation  when  entering  PBL  environment.  The  most  effective  way  for  an 
implementation  is  making  use  of  others’  experience. 

Not  only  partners’  employees  but  also  acquisition  personnel  of  buyer  should  be  aware  of 
what  PBL  is.  Subject  matter  experts  support  a  continuous  education,  training  for  PBL.  The  more 
you  learn  the  better  job  you  do.  As  stated  above,  this  is  a  culture  change  and  for  a  culture  change 
managements’  commitment  is  required. 


Depending  on  this  information  7  questions  are  asked  to  the  user.  Table  21  lists  the 
questions. 

Table  21.  Questions  for  PBL  Experience,  Awareness  and  Training. 


No 

Question 

1 

How  experienced  is  the  1st  partner  (organic)  in  PBL? 

2 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

3 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

4 

How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

5 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 

6 

How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

7 

How  committed  is  the  managements  for  the  culture  change? 

Each  question  has  0-10  score  range.  0  means  not  at  all  and  10  means  extremely.  The  first 
four  questions  ask  the  PBL  experience  level  of  all  partners  depending  on  the  number  of  partners. 
User  should  give  an  average  level  for  the  4th  question  for  4  and  more  partners.  Organization  itself 
is  a  partner,  too.  The  evaluation  score  for  this  factor  has  two  equally  weighted  elements;  average 
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of  partner’s  experience  by  means  first  4  questions  answers  and  average  of  other  sub  factors  by 
means  last  3  questions  answers. 

This  factor’s  weight  of  occurrences  will  be  in  the  middles,  too. 

#  of  occurances  in  the  literature  sample  34 

W  occurance,  8  tTTT  7  TTT  I  I  TrTT  7.07% 

'  E#  of  all  factors  occurances  481 

To  make  PBL  successful,  a  well-implemented  culture  change  is  required.  Therefore  it 
should  have  more  importance,  more  weight.  A  slightly  larger  weight  is  given  by  author. 

w  author  ,8  —  9% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

wuser,8  =  will  be  determined  by  user 

9.  Innovation 

Innovation,  creativity  is  a  fundamental  element  of  PBL,  especially  regarding  performance 
metrics  improvement.  Buyers’  intent  for  flexibility  is  almost  just  for  innovation.  Innovation  is 
another  skill,  that  contractor  should  have  in  addition  to  his/her  knowledge  and  experience.  S/he 
should  use  the  given  freedom  for  innovative  approach  to  have  better  results  regarding  metrics. 
Literature  is  full  of  encouragements  for  innovation  especially  in  DoD.  Also  subject  matter 
experts  agree  that  innovation  is  a  success  key  booster  in  PBL. 

Although  literature  gives  bunch  of  examples  and  states  much  importance  on  this  factor,  it 
does  not  give  insight  how  to  assess  innovation.  For  the  tool;  a  summary  of  Dyer,  Gregersen  and 
Christensen’s  (2011)  questions  and  survey  is  used.  According  to  them  innovative  organizations 
have  two  skills;  discovery  and  delivery.  PBL’s  innovation  is  dealing  with  discovery  part,  because 
delivery  part  just  includes  delivering  the  innovative  products/solutions.  They  also  figured  how  an 
innovative  organization  should  be.  Figure  10  shows  how  “innovative  companies  build  the  code 
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for  innovation  right  into  the  organization’s  people,  processes,  and  guiding  philosophies.”  (Dyer 


et  ah,  2011,  p.170) 


People 


Senior  executive(s)  lead 
the  innovative  charge  and 
excel  at  discovery  (discovery 
quotient  >75%). 

Monitor  and  maintain  an 
adequate  proportion  of  high- 
discovery-quotient  people  in  every 
management  level,  functional  area, 
and  decision-making  stage  of  the 
innovation  process. 


Philosophies 


■  Processes  explicitly 
encourage  employees  to 
associate,  question, 
observe,  network,  and 
experiment. 

■  Processes  are  designed 
to  hire,  train,  reward, 
and  promote  discovery- 
driven  people 


•  Philosophy  1 :  Innovation  is 
everyone’s  job — not  just  R&Ds. 

•  Philosophy  2:  Disruption  is  part 
of  our  innovation  portfolio. 

•  Philosophy  3:  Deploy  small, 
properly  organized  innovation 
project  teams. 

•  Philosophy  4:  Take  smart  risks 
in  pursuit  of  innovation. 


Figure  10.  Innovative  Organization  Structure  (Dyer  et  ah,  2011,  p.170). 

Those  are  three  elements  of  organization  structure.  According  to  this,  10  question  survey 
is  created  by  them.  A  summarized  version  of  this  survey  and  a  similar  scale  are  used  to  evaluate 
innovation. 

Evaluation  of  this  factor  includes  equally  weighted  two  parts;  one  of  them  assesses 
innovation  regarding  the  information  at  the  beginning  of  this  sub-section,  the  other  is  the  adopted 
survey.  Table  22  lists  the  questions. 

The  first  3  questions  are  for  the  first  part,  others  are  for  the  second.  Each  question  has  0- 
10  score  range.  0  means  not  at  all  and  10  means  extremely.  Scores  of  each  part  and  allover  score 
are  the  average  of  given  answers.  The  only  exception  regarding  scoring  is  the  first  question.  It’s 
converted  automatically  by  the  tool  due  to  its  negative  effect.  User  should  answer  them 
according  to  his/her  knowledge  about  candidates.  User  should  decide  how  to  assess  candidates 
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when  answering  questions.  If  the  answer  is  unknown,  user  may  select  N/A  and  the  average  does 
not  include  that  question/answer. 


Table  22.  Questions  for  Innovation. 


No 

Question 

1 

How  constraining  are  the  terms  blocking  innovational  approaches? 

2 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

3 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

4 

People:  How  capable  are  the  employees  regarding  well-  known  record  of  generating  innovative  ideas  in  this  partnership? 

5 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  this  partnership? 

6 

People:  How  important  will  innovation  be  as  a  lactor  at  employees'  performance  evaluations  in  this  partnership? 

7 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this  partnership? 

8 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this  partnership? 

9 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers,  competitors,  suppliers,  etc.? 

10 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for  new  ideas? 

11 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or  experiments? 

12 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative  ideas? 

13 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

Literature  emphasized  on  this  factor  sufficiently. 

#  of  occurances  in  the  literature  sample 


28 


w, 


occurance,  9 


=  5.82% 


T#  of  all  factors’  occurances  481 

Innovation  is  a  success  key  especially  for  improving  metrics.  A  similar  weight  is  given 
by  author. 


w  author,  9  6% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

w user, 9  =  w HI  be  determined  by  user 

10.  Data  Requirement 

PBL  is  buying  performance  according  to  in  advance-set  perfonnance  values  defined  in 
the  contract.  To  set  a  value,  past  experience,  historical  data  is  required.  ‘To  obtain  such  data’  is 
almost  impossible  for  new  systems.  Some  authors  determined  that  2  years’  of  data  is  required  for 
the  success  of  a  legacy  system  PBL.  Not  having  accurate,  adequate  data  will  increase  risk,  shared 
by  partners.  Data  is  needed  for  historical  and  present  performance  and  also  for  cost  estimation. 
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Present  data  is  required  to  assess  the  performance  metrics,  calculate  the  payment.  Buyer 
must  have  the  ability  to  reach  the  data  at  any  given  time.  These  procedures  should  not  be  blocked 
by  any  progress.  In  addition  to  that  buyer  should  have  the  necessary  tools  to  keep  track. 
Depending  on  this  information  1 1  questions  are  asked  to  the  user.  Table  23  lists  the  questions. 


Table  23.  Questions  for  Data  Requirement. 


No 

Question 

1 

How  adequate  is  historical  data? 

2 

How  accurate  is  historical  data? 

3 

How  adequate  is  cost  data? 

4 

How  accurate  is  cost  data? 

5 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

6 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

7 

How  many  years  of  performance  data  is  in  hand? 

8 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

9 

How  timely  is  the  data  management  as  a  level  of  real  time? 

10 

How  adequate  are  the  data  collection  tools? 

11 

How  sufficient  is  the  market  for  cost  estimation? 

Each  question  except  seventh  has  0-10  score  range.  0  means  not  at  all  and  10  means 
extremely.  Seventh  question  asks  the  user  to  determine  how  many  years  of  data  are  in  hand.  2 
years  and  more  data  will  get  10,  others  will  get  proportional  points.  User  should  give  answers  to 
all.  The  evaluation  score  will  be  average  of  the  answers. 

Literature  gives  sufficient  emphasis  on  this  factor. 

#  of  occurances  in  the  literature  sample  27 

^  occurance,  10  TTS  7  iT~c  I  I  "777T  5.61% 

T#  of  all  factors  occurances  481 

A  slightly  lower  weight  is  given  by  author. 


wauthor,  10  5.5% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

wuser, io  —  vvi/Z  be  determined  by  user 
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11.  Competition  and  PSI  Candidates 

Competition  is  important  for  PBL  as  for  other  acquisition  methods.  It  drives  candidates  to 
be  cheaper  and/or  better.  PBL’s  one  of  the  purposes  is  also  maximizing  competition  with 
provided  flexibility,  freedom.  It’s  mandated  by  regulations,  policies.  There  should  not  be  any 
block  to  prevent  competition.  Policies  also  points  to  consider  alternative  systems  to  meet 
requirements.  Buyer  should  think  globally;  qualified  international  sources  should  have  the 
opportunity  to  be  a  candidate. 

One  of  the  most  important  things  for  competition  is  willingness.  Buyer  should  find 
contractors  who  are  willing  to  accept  the  PBL  strategy. 

Literature  mentions  about  possible  candidates  for  PBL  and  also  mentions  about  their 
advantages/disadvantages  regarding  their  experience,  knowledge,  etc.  But  this  is  a  subject  of 
‘skills  of  partners,  best  practices’  factor.  But  there  may  be  an  exception,  here.  In  case  of  having 
just  one  candidate  for  the  acquisition,  OEM  or  prime  vendor  can  achieve  the  performance 
requirements,  although  there  is  not  any  competition. 

It  is  hard  to  manage  many  suppliers.  Therefore  system  level  PBL  under  an  umbrella  with 
a  PSI  is  better  for  success.  Experiences  show  that;  few  suppliers  are  good  for  PBL.  Buyer  would 
like  to  go  for  at  least  system  wide  PBL  programs.  Successful  PBL  examples  vary  within  1  to  4 
partners. 

Depending  on  this  information  6  questions  are  asked  to  the  user.  Table  24  lists  the 
questions. 

First  question’s  answer  is  converted  due  to  negative  effect.  Second,  third  and  fifth 
questions  are  ‘Yes/No’  questions.  ‘Yes’  will  add  1  more  point  to  the  overall  score;  ‘No’  will  not 
add  any.  Fourth  question  has  a  scale  of  0-10.  0  means  not  at  all  and  10  means  extremely.  Last 
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question  is  the  number  of  possible  partners  including  the  buyer.  Answers;  2,  3,  4,  5,  6  or  more 
will  get  10,  8,  6,  4,  2,  0  respectively.  Overall  score  for  evaluation  of  this  factor  is  calculated  by 
taking  average  of  questions  1,  4,  6  and  adding  the  points  gained  by  questions  2,  3  and  5.  If  the 
result  is  larger  than  10,  the  tool  will  not  give  more  than  10. 


Table  24.  Questions  for  Competition  and  PSI  Candidates. 


No 

Question 

1 

How  constraining  are  the  terms  blocking  competition? 

2 

Are  alternative  systems  considered  to  meet  requirements? 

3 

Are  qualified  international  sources  permitted  to  enter? 

4 

How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 

5 

In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 

6 

How  many  partners  are  present  in  PBL  including  buyer? 

More  than  25  percent  of  the  sample  literature  emphasized  on  this  factor. 

#  of  occurances  in  the  literature  sample  24 

w  ,  =  — - - - —  = - —  4  QQo/n 

occurance,  £#  factors’  occurances  481 

Literature  gives  more  importance  than  it  is  required  by  this  factor.  All  acquisition 

strategies  need  competition  somehow.  A  lower  weight  is  given  by  author. 

w author,  11  ~  3.5% 

As  mentioned  before  user  may  give  his/her  own  weight  to  this  factor: 

wuser, ii  —  wi/Z  be  determined  by  user 

12.  Properties  of  System 

Literature  concludes  not  every  acquisition  fits  in  PBL.  PBL  is  mostly  good  for  new, 
strategically  important,  high-tech,  complex,  long  life,  capital  intensive  systems.  And  PBL  is  not 
good  at  all  either  for  legacy  systems  at  their  end  of  lives,  or  for  highly  reliable  systems.  But  PBL 
is  recommended  for  capital  intensive  systems.  If  insourcing  required  lots  of  investments,  PBL 
would  rather  be  preferred. 
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Depending  on  this  information  7  questions  are  asked  to  the  user.  Table  25  lists  the 


questions. 


Table  25.  Questions  for  Properties  of  System. 


No 

Question 

1 

Is  contract's  subject  a  new  goods/service? 

2 

Howcrtitical  is  the  goods/service  regarding  strategic  importance? 

3 

How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 

4 

How  complex  is  the  goods/service? 

5 

What  is  the  percentage  of  expected  life  left  on  goods/service? 

6 

How  reliable  is  the  goods/service? 

7 

How  much  investment  is  required  for  insourcing? 

First  question  is  a  ‘Yes/No’  questions.  ‘Yes’  will  add  1  point  to  the  overall  score;  ‘No’ 
will  not  add  any.  Others  except  fifth  question  have  0-10  point  scale.  Fifth  one  needs  an  answer  in 
percentage.  100%  gets  10  points.  Only  sixth  question  has  a  negative  effect  and  it  is  converted 
automatically  in  calculations.  Overall  score  will  be  the  sum  of  last  6  answer’s  average  and 
‘Yes/No’  answer.  If  the  score  goes  over  10  due  to  this  summation,  the  tool  will  give  10 
maximum. 

Literature  does  not  emphasize  this  factor  much. 

#  of  occurances  in  the  literature  sample  17 

W occur  ance,  12  tTTT  7  iT~c  I  I  "777T  3.53% 

27#  of  all  factors  occurances  481 

This  is  a  dominant  factor  to  detennine  an  acquisition  method.  On  the  other  hand  it  is  not 
mentioned  a  lot  in  the  literature,  although  in  some  cases  authorities  emphasized.  A  higher  weight 
is  given  by  author. 


w author, 12  ~  6% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor: 

wuser, i2  —  wi/Z  be  determined  by  user 
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13.  Up-Front  Planning  and  Early  PBL  Implementation 

Early  decision  of  PBL,  will  booster  success  of  the  sustainment  program.  Owner  wants  to 
keep  the  system  ready.  Therefore  support  strategies  should  be  considered  even  at  the  decision 
making  progress  of  buying  a  new  system.  As  seen  this  factor  is  only  applicable  for  new  systems. 

Early  implementation  includes  well  implementation  of  sustainment  processes  regarding 
design,  sustainment  in  the  field,  partnership  and  responsibilities. 


Depending  on  this  information  6  questions  are  asked  to  the  user.  Table  26  lists  the 
questions. 

Table  26.  Questions  for  Early  Planning  of  PBL. 


No 

Question 

1 

Is  'PBL  decision'  given  early  in  acquisition  of  system? 

2 

Is  'sustainment  of  the  system  as  it  is  being  designed'  well-planned  up- front? 

3 

Is  'service  of  the  system  serviced  in  the  field'  well-planned  up-front? 

4 

Is  'partnering  of  industry  and  service  branch  support  personnel  in  sustaining  that  asset'  well-planned  up-front? 

5 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? 

All  questions  are  ‘Yes/No’  questions.  ‘Yes’  represents  10  points  and  ‘No’  0.  Overall 
score  has  two  elements  due  to  their  importance;  the  first  answer  and  the  average  of  last  five 
questions.  User  should  answer  them  all.  These  questions  are  not  applicable  for  legacy  system 
PBLs. 


Literature  emphasizes  this  factor  well. 


woccurance,  13 


#  of  occurances  in  the  literature  sample 
T#  of  all  factors’  occurances 


16 

481 


3.33% 


A  similar  weight  is  given  by  author. 


w author,  13  3% 

As  mentioned  before  the  user  may  give  his/her  own  weight  to  this  factor. 
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wuser,  13  —  wiZZ  be  determined  by  user 


*  , i  A  ■  r  A  ■  r  ■  Table  27.  Weights  for  Legacy  Systems. 

Another  effect  ot  this  tactor  is  changing  °  °  J  J 

the  weights  of  all  factors  in  case  of  deciding 

PBL  for  a  legacy  system.  Because  this  factor  is 

not  applicable  to  new  systems.  Total  number  of 

all  factors’  occurrences  goes  from  481  to  465. 

Table  27  lists  the  new  weights  for  occurrences’ 

frequency  and  author’s  determination,  which 

will  be  applied  to  legacy  system  PBL 

evaluations  automatically  by  the  tool. 

14.  General  Score  and  Specifications 

The  tool  calculates  each  factor’s  score  and  state  them  in  the  results  spreadsheet.  General 

score  is  the  total  of  each  score.  According  to  the  author  weighted  tool  score;  tool  states  a 

percentage  of  how  well  PBL  fits  this  acquisition  and  also  made  a  comment  for  a  better 

understanding.  Comments  are  listed  with  their  corresponding  scores  in  table  28. 


Factor# 

Weight 

Occu  ranees 

Author 

1 

12.90% 

6.00% 

2 

12.04% 

13.00% 

3 

11.18% 

13.00% 

4 

10.54% 

11.00% 

5 

8.82% 

9.00% 

6 

8.39% 

10.00% 

7 

8.17% 

8.00% 

8 

7.31% 

9.00% 

9 

6.02% 

6.00% 

10 

5.81% 

5.50% 

11 

5.16% 

3.50% 

12 

3.66% 

6.00% 

TOTAL 

1.00 

1.00 

Table  28.  Comments  of  the  Tool. 


Score 

Comment 

[0-1] 

Definetely  do  not  use  PBL  as  an  acquisition  strategy. 

(1-4] 

PBL  might  not  be  the  best  acquisition  strategy. 

(4-6] 

PBL  might  me  considered  as  an  acqusition  strategy. 

But  detailed  Business  Case  Analysis  is  required. 

(6-8] 

PBL  is  advised  as  an  acquisiton  strategy. 

(8-10] 

Definitely  use  PBLas  an  acquisition  strategy. 

In  addition  to  those  comments  if  author  weighted  tool  score  is  below  8,  tool  points  out 
which  factors  should  be  paid  more  attention  for  a  better  PBL. 
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The  tool  has  several  and  variety  of  logical  controls.  These  will  help  user  in  case  of  wrong 
selection.  Results  sheet  will  state  which  portion  of  evaluation  form  should  be  corrected  by 
pointing  out  number  of  factor. 

An  instructions  sheet  is  implemented  in  the  tool.  To  reach  that  sheet  a  button  is  present  at 
the  evaluation  form  sheet.  User  may  use  these  linked  buttons  to  reach  the  desired  sheet.  Three 
linked  buttons  are  present  for;  evaluation  form,  instructions  and  result.  There  is  another  button  in 
the  evaluation  form  called  reset.  This  is  a  macro  which  resets  all  the  cells  to  their  original  value. 
There  are  also  up-arrow  buttons,  which  might  be  used  to  go  top  of  the  selected  sheet. 

The  cells  which  are  not  for  use  of  users  are  locked  to  prevent  any  wrong  selection,  fault. 
Data  spreadsheet,  which  has  the  calculations  in  it,  is  hidden  for  same  purposes.  Gridlines, 
headings  and  unused  cells  are  hidden  for  a  better  view.  Cells  are  filled  in  colors  for  same 
purposes. 
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IV.  Application,  Results,  and  Analysis 

In  this  chapter  the  tool,  which  is  created  in  methodology,  is  applied  to  5  different 
acquisitions  and  the  results,  analysis  are  represented.  These  acquisition  examples  are: 

1 .  Mass  transportation  of  Turkish  Armed  Forces  personnel 

2.  Meal  service  for  privates 

3.  ANKA  Turkish  Unmanned  Aerial  Vehicle  Support  Program 

4.  Turkish  Army’s  advanced  Annored  Combat  Vehicle  (ACV)  Support  Program 

5.  F-35  Joint  Strike  Fighter  (JSF)  Support  Program 

First  two  examples  were  also  suggested  by  Bozkurt  and  Guducu  (2005)  in  their  study  for 
Turkish  Navy.  ANKA  is  the  first  domestically  manufactured  Unmanned  Aerial  Vehicle  which  is 
planned  for  mass-production  in  the  near  future.  Implementing  PBL  for  Turkish  Army’s  advanced 
ACV  is  Denizer’s  (2007)  study.  And  one  of  the  partners  of  F-35  JSF  project  is  Turkish  Air 
Force.  The  support  program  is  planned  to  be  PBL. 

The  tool  created  at  methodology  is  fulfilled  according  to  corresponding  acquisition 
subject  matter  experts’  and/or  program  managers’  ideas,  contributions,  and  comments.  For  each 
application  the  acquisition  is  explained  briefly  and  the  results  of  the  tool  accompanying  by 
expert’s  ideas  and  analysis  are  given.  Due  to  confidentiality  reasons;  their  names,  units  are  not 
given  and  some  of  the  data  are  disguised  within  the  range  of  evaluation  score. 

The  tool  and  analysis  in  this  research  was  created/perfonned  on  a  personal  computer  with 
an  Intel(R)  Core(TM)  i7  CPU  Q2670QM  @2.20  GHz  Processor,  8  GB  RAM,  Windows  7  Home 
Premium  64-bit  operating  system,  Microsoft  Office  Home  and  Student  2010. 
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1.  Mass  transportation  of  Turkish  Armed  Forces  Personnel 

Transportation  Unit  A  is  responsible  for  nearly  7,000  personnel’s  daily  transportation  in 
one  of  the  pretty  populated  cities  in  Turkey,  which  has  an  area  of  nearly  2,300  square  miles. 

There  are  approximately  80  different  routes.  The  unit  has  three  sub-units.  Two  of  them  are 
responsible  for  50%  of  transportation  and  the  other  is  responsible  for  other  part  which  is 
outsourced  to  a  third  party.  Due  to  lack  of  organic  capabilities;  most  of  the  mass  transportation  of 
personnel  is  done  by  contracted  third  parties.  In  this  example  Unit  A  plans  to  increase  contracted 
transportation  service  percentage  from  50%  to  70%.  30  %  will  remain  organic.  And  the  tool  will 
help  us  to  decide  whether  PBL  fits  in  this  acquisition  or  not. 

Three  metrics  are  planned  to  be  used;  on  time  transportation  to  last  stop;  timely  arrival  to 
each  stop  and  passenger  satisfaction  regarding  the  condition  of  vehicles,  driving  habits  of 
drivers,  etc.  These  metrics  are  determined  by  unit  and  not  negotiated  with  candidates.  According 
to  one  of  the  PMs;  to  accomplish  the  mission  is  more  important  than  cost  issues  although  there 
are  some  budget  constraints.  But  they  are  expecting  a  similar  cost  reduction  like  they 
experienced  when  they  had  gone  outsourcing.  One  of  the  purposes  for  outsourcing  is  warrant 
qualified  drivers,  service  to  hard-to-drive  areas  which  has  several  blind  spots  in  that  region.  Till 
now,  contactors  in  that  city  have  done  a  pretty  good  job.  There  are  not  any  political  barriers 
though  there  are  regulations,  constraining  tenns,  general  military  rules,  etc.  And  also  more  than 
one  year  contracting  is  prohibited  by  acquisition  regulations.  Exceptions  are  not  expected  in  this 
case.  Organic  structure  has  qualified  personnel  and  tools  for  performance  data  collection,  besides 
they  have  almost  5  years  of  historical  data.  But  neither  of  the  possible  partners  including  the 
organization  itself  is  not  experienced  or  trained  in  PBL.  Generally  8  firms  are  willing  to  be 
awarded  for  this  contract.  Three  of  them  are  big  firms  in  this  area.  Till  now  most  of  the  contracts 
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awarded  to  two  firms.  One  of  the  firms  is  responsible  for  60%  of  the  outsourced  area  and  the 
other  for  40%. 

The  tool  was  fulfilled  by  PM;  evaluation  and  results  are  presented  in  Appendix  C. 
Results'  summary  is  presented  in  figure  1 1 . 


According  to  author's  weighted  tool  score  PBL  fits  for  this  acqisition  by  31.9  %.  General  Scores: 

PBL  might  not  be  the  best  acquisition  strategy.  Business  case  analysis  is  advised. 

To  have  a  more  successfol  PBL  pay  more  attention  to  improve  the  factors  below:  Occurances'  weighted  Tool  Score  3.1356 

Reduction  in  total  cost  of  ownership,  life  cycle  cost;  Presence,  Determination  of  metrics,  performance  Author's  weighted  Tool  Score  3.1886 

outcomes;  Improvement  in  performance;  Partnership,  strategic  alliance;  Length  of  contract;  User's  weighted  Tool  Score  3.6804 

Flexibility;  Better  Service,  Best  Value  Created,  Best  Practices  ;  Successful  implementation  of  PBL, 

PBL  experience  and  training;  Innovation;  Data  availability;  Candidates  and  competition;  Properties  of  Occurances'  weighted  User's  Score  2.8108 

system;  .  Author's  weighted  User's  Score  2.9650 

_ User's  weighted  User's  Score _ 3.5287 


Figure  11.  Results  for  Mass  transportation  of  Turkish  Armed  Forces  Personnel. 

As  seen  PBL  fits  in  by  around  32%.  Therefore  PBL  is  not  advised  since  it  might  not  be 
the  best  acquisition  method.  Organization  should  find  a  way  to  improve  the  factors;  reduction  in 
total  cost  of  ownership,  life  cycle  cost;  presence,  determination  of  metrics,  performance 
outcomes;  improvement  in  performance;  partnership,  strategic  alliance;  length  of  contract; 
flexibility;  better  service,  best  value  created,  best  practices;  successful  implementation  of  PBL, 
PBL  experience  and  training;  innovation;  data  availability;  candidates  and  competition; 
properties  of  system. 

In  such  a  constraining  environment  and  lack  of  long  contracts  due  to  regulations;  the 
result  is  expected.  Although  some  of  the  authors  recommend  using  PBL  in  these  services,  it  does 
not  seem  to  be  the  best  way  for  procurement.  PBL  is  not  just  outsourcing,  indeed.  Although 
user’s  weighted  score  is  close,  there  is  a  10.66%  positive  difference  is  present.  Figure  12  shows 
the  differences  at  each  factor  evaluation  and  also  for  author  weighted  tool  score  and  user 
weighted  user  score.  As  seen;  subjectivity  may  cause  so  much  variability  in  evaluation. 
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Figure  12.  Subjectivity  Analysis  for  Mass  transportation  of  Turkish  Anned 
Forces  Personnel. 


2.  Meal  Service  for  Privates 

In  Turkey  military  service  for  1.5  years  is  mandatory.  All  expenses  regarding  subsistence, 
quarter,  clothes,  apparel,  etc.,  are  provided  by  Government.  Turkish  Quartermaster  officers  are 
responsible  for  these  types  of  acquisitions.  There  is  a  trend  for  third  party  logistics  in  meal 
services.  Bozkurt  and  Guducu  (2005)  also  suggested  using  PBL  in  this  case. 

Unit  B  is  responsible  for  3,000  privates’  meal  service  with  around  75  personnel.  There 
are  totally  five  kitchens  which  serves  13  different  locations.  The  meal  service  consists  of  3 
meals;  breakfast,  lunch  and  dinner.  Three  meals  should  include  at  least  3,500  calories  daily.  Unit 
B  is  planning  to  have  a  service  contract  for  a  year.  Maximum  one  year  contract  is  available 
according  to  regulations.  They  are  expecting  a  5%  total  cost  reduction  and  a  better  service 
quality.  Some  of  the  facilities  will  be  allowed  to  be  used  by  contractor.  But  there  are  certain  strict 
regulations  not  only  in  procedures  but  also  in  providing  meals,  ‘what  to  do’s.  For  a  PBL,  calories 
of  meals  limited  to  optimum,  customer  satisfaction,  the  accordance  to  technical  specifications 
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document,  on  time  service,  sharing  benefits  in  cost  reduction,  number  of  staff  in  service  are 
planned  to  be  used  as  metrics.  There  are  several  firms  willing  to  enter  although  neither  of  them  is 
experienced  in  PBL.  But  they  are  doing  pretty  well  job  in  this  business. 

Tool  is  fulfilled  according  to  PM's  knowledge.  Evaluation  and  results  are  presented  at 
Appendix  C.  Results'  summary  is  presented  in  figure  13. 


According  to  author's  weighted  tool  score  PBL  fits  for  this  acqisition  by  22.6  %.  General  Scores: 

PBL  might  not  be  the  best  acquisition  strategy.  Business  case  analysis  is  advised. 

To  have  a  more  successful  PBL  pay  more  attention  to  improve  the  factors  below:  Occurances'  weighted  Tool  Score  2.2329 

Reduction  in  total  cost  of  ownership,  life  cycle  cost;  Presence,  Determination  of  metrics,  performance  Author's  weighted  Tool  Score  2.2550 

outcomes;  Improvement  in  performance;  Partnership,  strategic  alliance;  Length  of  contract;  User's  weighted  Tool  Score  3.1413 

Flexibility;  Better  Service,  Best  Value  Created,  Best  Practices  ;  Successful  implementation  of  PBL, 

PBL  experience  and  training;  Innovation;  Data  availability;  Properties  of  system; .  Occurances'  weighted  User's  Score  2.2366 

Author's  weighted  User's  Score  2.2450 

_ User's  weighted  User's  Score _ 2.8841 


Figure  13.  Results  for  Meal  Service  for  Privates. 

Although  the  service  which  will  be  provided  by  contractor  is  better,  the  constraining  of 

regulations,  rules  do  not  give  any  improvements.  Since  the  organic  structure  does  a  pretty  well 

job  in  some  of  the  metrics,  there  are  not  much  improvement  is  expected.  Therefore  according  to 

the  results  PBL  is  not  a  good  option.  It  fits  22.6%,  so  close  to  the  lowest  level  20%.  Although 

user’s  weighted  score  is  close,  there  is  a  27.89%  positive  difference  is  present.  Figure  14  shows 

the  differences  at  each  factor  evaluation  and  also  for  author  weighted  tool  score  and  user 

weighted  user  score.  As  seen;  subjectivity  may  cause  so  much  variability  in  evaluation. 


Figure  14.  Subjectivity  Analysis  for  Meal  Service  for  Privates. 


3. 


ANKA  Turkish  Unmanned  Aerial  Vehicle  Support  Program 


Figure  15.  ANKA,  Turkish  UAV  (TAI,  2012). 


ANKA  is  an  UAV  developed  by  Turkish  Aerospace  Industries  (TAI),  for  intelligence, 
surveillance  target  acquisition  and  reconnaissance  (TAI,  2011).  The  name,  ANKA  comes  from  a 
mythical  flying  creature  Phoenix.  Technical  specifications  are  listed  in  table  29. 


Table  29.  Technical  Specification  for  UAV  ANKA  (Adopted  from  TAI,  2011). 


Specifications 

Unit 

Value 

Specifications 

Unit 

Value 

Service  Ceiling 

ft 

30000  MSL 

Fuselage  Length 

m 

10 

Maximum  Endurance 

hrs 

24 

Wing  Span 

m 

17 

Cruise  Speed 

kts 

110 

Wing  Area 

2 

m 

13.6 

Engine  Power 

hp 

115  ISA@SL 

Wing  Aspect  Ratio 

- 

22 

Datalink  Range 

mi 

200 

Wing  Sweep  (quarter  chord) 

O 

0 

Payload  Weight 

kg 

200 

Environmental  Conditions:  15  kts  side  wind, 

20  kts  head  wind;  temperature,  humidity, 
rain  and  icing  limits  for  MIL-HDBK-310 

Fuel  Weight 

kg 

300 

Maximum  Take-Off  Weight 

kg 

1600 

The  contract  for  an  indigenous  Medium  Altitude  Long  Endurance  (MALE)  Unmanned 
Aerial  Vehicle  (UAV)  system  was  signed  between  Turkish  Air  Force  and  TAI  on  24  December 
2004. 


“Within  the  framework  of  the  program,  a  total  of  three  prototypes  and  ground  systems 
will  be  designed,  developed,  manufactured  and  tested  by  mid-201 1  as  part  of  the 
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prototype  development  phase.  Subsequently  in  2012,  the  serial  production  phase  of 
Anka-A  will  be  launched  and  additional  10  systems  (meaning  30  air  vehicles)  will  be 
built  for  the  Turkish  Air  Force.  To  attain  an  indigenous  power  unit  for  the  Anka,  Tusas 
Engine  Industries  (TEI)  develops  a  turboprop  engine  expected  to  start  trials  in  2011.” 
(Wikipedia,  TAI  Anka,  2012) 

After  several  inspections,  tests  and  developments  Defence  Industry  Executive  Committee 
contracted  Turkish  Aerospace  Industries  for  the  serial  production  of  10  ANKA  vehicles  on 
January  5,  2012. 

MICAP  ratios  and  fleet  readiness  are  planned  to  be  metrics.  The  expectations  are  similar 
to  Shadow  Unmanned  Aerial  Vehicle  PBL  program  in  U.S.  There  are  regulations  blocking 
innovations  approaches  and  flexibility.  An  exception  is  planned  to  be  approved  from  the  Turkish 
Grand  National  Assembly  for  this  program  regarding  these  regulations.  Although  this  is  a  new 
system,  Air  Force  maintenance  personnel  is  skilled,  experienced  in  similar  systems.  But  neither 
organic  nor  commercial  structures'  personnel  are  experienced  in  PBL.  This  PBL  program  is 
planned  to  be  at  least  a  5-years  contract  and  budgeted  in  10  year  procurement  plans.  Since  the 
system  is  domestically  designed,  manufactured,  especially  commercial  structure  may  be  counted 
as  innovative. 

The  tool  is  fulfilled  by  one  of  the  knowledgeable  acquisition  officer’s  comments  in  this 
project.  Evaluation  and  results  are  presented  in  Appendix  C.  Results'  summary  is  presented  in 


According  to  author's  weighted  tool  score  PBL  fits  for  this  acqisition  by  61.5  %.  General  Scores: 

PBL  is  advised  as  an  acquis iton  strategy.  Business  case  analysis  is  required. 

To  have  a  more  successful  PBL  pay  more  attention  to  improve  the  factors  below:  Occurances'  weighted  Tool  Score  6.0659 

Reduction  in  total  cost  of  ownership,  life  cycle  cost;  Presence,  Determination  of  metrics,  performance  Author's  weighted  Tool  Score  6.1462 

outcomes;  Improvement  in  performance;  Partnership,  strategic  alliance;  Length  of  contract;  User's  weighted  Tool  Score  6.3938 

Flexibility;  Better  Service,  Best  Value  Created,  Best  Practices  ;  Successful  implementation  of  PBL, 

PBL  experience  and  training;  Innovation;  Data  availability;  Up-front,  early  planning.  Occurances'  weighted  User's  Score  6.2765 

Author's  weighted  User's  Score  6.2800 

_ User's  weighted  User's  Score _ 6.3265 


Figure  16.  Results  for  ANKA  Turkish  UAV  Support  Program. 
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Since  the  system  is  domestically  produced,  environment  is  suitable  for  PBL.  The  only 
lack  is  experience  in  this  case.  This  program  fits  by  61.5%  and  gaining  experience  will  lead  to 
higher  percentages.  Although  user’s  weighted  score  is  close,  there  is  a  3%  positive  difference  is 
present.  Figure  17  shows  the  differences  at  each  factor  evaluation  and  also  for  author  weighted 
tool  score  and  user  weighted  user  score.  As  seen;  subjectivity  may  cause  so  much  variability  in 
evaluation. 


Figure  17.  Subjectivity  Analysis  for  ANKA  Turkish  UAV  Support  Program. 


4.  Turkish  Army’s  Advanced  Armored  Combat  Vehicle  (ACV)  Support  Program 

ACV  platforms  are  used  for  support  to  tanks  and  armored  personnel  carriers.  It  is  easy  to 
implement  variety  of  weapons  (such  as  machineguns,  antitank  weapons,  mortar,  cannon,  etc.)  on 
it.  Therefore  costs  are  reduced  by  multitasking.  These  are  saviors  of  Land  Forces  in  field.  Their 
speed  is  around  65  km./h.,  range  is  around  490  km.,  and  capacity  is  1 1  personnel.  These  systems 
are  not  only  used  by  Turkish  Land  Forces  but  also  exported  to  BAE,  Malaysia,  Philippines,  and 
Saudi  Arabia. 
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Figure  18.  Turkish  ACV  Platform  (SSM,  2012). 


Turkish  Defense  Industry  Executive  Committee  (DIEC)  awarded  FMS-NUROL 
consortium  twice;  once  for  1,698  ACV  platforms  in  1989  completed  in  2000,  and  once  for  551 
ACV  platforms  in  2000  completed  in  2005. 

Although  acquisition  program  followed  five  main  implementation  steps  (prototype 
production,  test  and  evaluation,  production,  acceptance,  evaluation  and  testing,  shipment  to  the 
warfighters,  guarantee  period)  domestically,  and  support  strategy  is  total  life  cycle  support 
management,  PBL  is  not  considered  at  all.  Turkish  Army  Logistics  Command  is  responsible  to 
make  them  MICAP. 

Denizer  (2007)  suggested  to  implement  PBL  strategy  to  ACV  platforms  likewise  U.S. 
Anny  did  in  Stryker  Interim  Combat  Vehicle  (ICV).  The  tool  is  used  to  assess  the  conformity 
according  to  his  study  and  open  sources. 

Regarding  competition,  skills  and  capabilities  of  contractors: 

“Turkey’s  industrial  sector  is  able  to  present  and  provide  competitive  logistic  capabilities 
especially  in  the  ground  vehicles  industry.  The  utilization  of  this  industrial  potential  for 
military  sustainment  may  be  beneficial  in  terms  of  not  only  getting  better  quality  service 
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support,  but  also  adopting  better  business  practices  used  successfully  by  the  commercial 
adversaries  to  the  military  logistics  over  a  certain  period.”  Denizer  (2007,  p.52) 

Partnering  opportunities  is  also  emphasized  in  his  study.  In  addition  to  that  he  declared  organic 

maintenance  capability,  training  as  one  of  the  bests. 

In  his  study  he  applied  the  12  steps  for  implementation  by  comparing  Stryker  ICV  and 

Turkish  ACV  in  details.  Although  each  step  is  seemed  to  be  applicable  to  ACVs,  he  admits  that 

PBL  is  a  new  term  for  Turkish  Armed  Forces.  According  to  him  “since  this  concept  is  quite  new 

for  Turkish  Army  logistics,  it  seems  to  be  beneficial  to  start  with  a  selective  pilot  project  to 

improve  experience  to  adapt  it  into  organization  and  culture.”  (Denizer,  2007,  p.90) 

He  expects  a  similar  cost  savings  as  U.S.  did  in  Strykers;  $12/$20  per  mile,  40% 

reduction.  But  he  also  admits  that  lack  of  data  issues  will  be  experienced  likewise  Stryker  PBL 

program  although  Turkish  Army  is  in  business  for  almost  10  years.  Top  level  five  metrics  are 

suggested  to  be  used  by  him  as  U.S.  did  in  Strykers  like  availability  and  reliability  metrics. 

The  tool  is  fulfilled  by  one  of  the  knowledgeable  acquisition  officers  in  this  project. 

Evaluation  and  results  are  presented  in  Appendix  C.  Results'  summary  is  presented  in  figure  19. 


According  to  author's  weighted  tool  score  PBL  fits  for  this  acqisition  by  59.4  %.  General  Scores: 

PBL  might  be  considered  as  an  acqusition  strategy.  But  detailed  Business  Case  Analysis  is  required. 

To  have  a  more  successtul  PBL  pay  more  attention  to  improve  the  factors  below:  Occurances'  weighted  Tool  Score  5.8678 

Presence,  Determination  of  metrics,  performance  outcomes;  Improvement  in  performance;  Author's  weighted  Tool  Score  5.9352 

Partnership,  strategic  alliance;  Length  of  contract;  Flexibility;  Successful  implementation  of  PBL,  PBL  User's  weighted  Tool  Score  6.2763 

experience  and  training;  Innovation;  Data  availability;  Properties  of  system; . 

Occurances'  weighted  User's  Score  5.8323 

Author's  weighted  User's  Score  6.2250 

User's  weighted  User's  Score  6.4405 


Figure  19.  Results  for  Turkish  ACV  Support  Program. 

This  is  also  a  domestically  produced  system.  Experience  and  skilled  force  is  present  in 

service.  The  only  limitation  is  experience  in  PBL.  Although  results  show  that  PBL  might  be 

considered  as  an  acquisition  strategy  by  59.4%,  since  the  next  level  is  60%  this  support  program 

will  be  considered  a  successful  PBL.  This  time  user's  weighted  score  is  a  little  bit  different  than 

author  weighted  tool  score.  The  reason  might  be  the  willing  to  use  PBL  as  an  acquisition  method. 

There  is  an  8.51%  positive  difference  is  present.  Figure  20  shows  the  differences  at  each  factor 
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evaluation  and  also  for  author  weighted  tool  score  and  user  weighted  user  score.  As  seen; 
subjectivity  may  cause  so  much  variability  in  evaluation. 


Figure  20.  Subjectivity  Analysis  for  Turkish  ACV  Support  Program. 


5.  F-35  Joint  Strike  Fighter  (JSF)  Support  Program 


Figure  21.  F-35  Joint  Strike  Fighter.(JSF,  2012). 


F-35  is  the  next  generation  strike  aircraft  weapon  systems  which  bring  last  technologies 
to  the  battle  space  of  the  future.  JSF  program  excelled  among  various  tactical  aircraft  acquisition 
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programs.  But  the  experience,  which  was  gained  from  other  programs,  is  precious.  This  aircraft 

is  planned  to  be  a  very  reliable  weapon  system. 

“The  JSFs  advanced  airframe,  autonomic  logistics,  avionics,  propulsion  systems,  stealth, 
and  firepower  will  ensure  that  the  F-35  is  the  most  affordable,  lethal,  supportable  and 
survivable  aircraft  ever  to  be  used  by  so  many  warfighters  across  the  globe.”  (JSF,  2012) 
This  is  an  international  program  consists  of  9  different  countries  and  one  of  them  is 

Turkey  by  nearly  4%  stake.  U.S.  Department  of  Defense  awarded  Lockheed  Martin  for  this 

program  in  2001.  Lockheed  Martin  is  the  F-35  prime  contractor,  while  Northrop  Grumman  and 

BAE  Systems  are  partners. 

In  201 1,  972  flights  are  flown  and  7,823  test  points  are  tallied.  This  is  an  achievement 
beyond  plans. 

“. . .  flight  test  program  plan  calls  for  the  verification  of  59,585  test  points  through 
developmental  test  flights  by  Dec.  3 1,  2016.  Through  2011,  the  flight  test  team  has 
accomplished  12,728  test  points  or  21.4  percent  of  overall  testing  requirements.”  (JSF, 
2012) 

This  program  keeps  aircrafts  MICAP  whenever  required.  It  was  designed  for  optimum 
sustainability.  The  platfo nil’s  sustainability  features  include  advanced  monitoring,  maintenance 
and  prognostics,  streamlined  service  operations,  autonomic  logistics  information  system,  global 
logistics,  key  supportability  benefits  (such  as  readiness  and  reliability,  force  protection  and 
mission  fulfillment,  cost  savings). 

The  program  is  a  well-known  example  of  up-front  planning  of  PBL.  Besides  all  war 
game  simulations  are  done  depending  on  PBL  support  decision.  The  partners  are  experienced  not 
only  in  their  job  but  also  in  PBL. 

The  tool  is  fulfilled  according  to  one  of  the  PMs's  ideas,  comments.  Although  Turkey  is 
willing  to  be  a  PBL  partner  in  this  program,  personnel  do  not  have  much  experience  and 
knowledge  about  PBL.  But  Air  Force's  organic  maintenance  capability  is  sufficient.  As  U.S.  did 
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in  previous  PBL  programs  MICAP  ratios,  fleet  readiness,  cost  reduction  are  planned  to  be  used 
as  metrics.  Cost  reduction  expectations  are  similar;  20%.  It  is  planned  to  be  at  least  a  5-years 
contract  and  budgeted  in  10  year  procurement  plans.  Political  and  budget  issues  have  been 
experienced  in  U.S.  regarding  JSFs.  Since  the  program  is  internationally  contributed,  these  kinds 
of  issues  may  occur  in  the  future,  too.  Although  Turkish  acquisition  regulations  do  not  allow 
what  PBL  requires,  there  will  be  exceptions  for  this  program  after  approval  of  the  Turkish  Grand 
National  Assembly. 

Evaluation  and  results  are  presented  in  Appendix  C.  Results'  summary  is  presented  in 
figure  22. 


According  to  author's  weighted  tool  score  PBL  fits  for  this  acqisition  by  70.5  %.  General  Scores: 

PBL  is  advised  as  an  acquis iton  strategy.  Business  case  analysis  is  required. 

To  have  a  more  successful  PBL  pay  more  attention  to  improve  the  factors  below:  Occurances'  weighted  Tool  Score  6.9132 

Reduction  in  total  cost  of  ownership,  life  cycle  cost;  Presence,  Determination  of  metrics,  performance  Author's  weighted  Tool  Score  7.0497 

outcomes;  Improvement  in  performance;  Partnership,  strategic  alliance;  Flexibility;  Better  Service,  User's  weighted  Tool  Score  7.0347 

Best  Value  Created,  Best  Practices  ;  Successful  implementation  of  PBL,  PBL  experience  and 

training;  Innovation;  Data  availability;  Up-front,  early  planning.  Occurances'  weighted  User's  Score  6.5821 

Author's  weighted  User's  Score  6.6800 

_ User's  weighted  User's  Score _ 6.5437 


Figure  22.  Results  for  JSF  Support  Program. 

PBL  is  advised  for  this  program.  The  only  limitation  is  experience  in  PBL  for  Turkey. 
Other  than  that  this  is  a  PBL  environment.  It  fits  70.5%,  but  it  will  be  increased  day  by  day  by 
experience  in  PBL.  There  is  a  7.17%  negative  difference  is  present.  Figure  23  shows  the 
differences  at  each  factor  evaluation  and  also  for  author  weighted  tool  score  and  user  weighted 
user  score.  As  seen;  subjectivity  may  cause  so  much  variability  in  evaluation. 

Validation  and  Verification  of  the  Tool 

The  results  show  that  the  tool  does  the  job,  it  is  pretended  to  do.  Therefore  it  can  be  stated 
that  the  right  model/tool  is  built.  All  of  the  results  are  expected  in  each  situation  and  each 
answer.  After  the  evaluation,  the  comments  and  suggestions  regarding  the  tool  are  asked  to  users, 
too.  Although  the  comments  are  mostly  positive,  the  negative  comments  are  taken  into  account 
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Figure  23.  Subjectivity  Analysis  for  JSF  Support  Program. 


and  the  tool  is  improved,  upgraded  according  to  their  ideas.  Therefore  a  more  user  friendly  tool 
is  built.  Positive  ideas  also  supported  that  the  tool  works.  In  addition  to  validation;  each  result 
according  to  the  answer  given,  is  double  checked  in  every  example  and  it  is  found  that  the 
model/tool  is  right.  Every  answer  is  converted  to  the  quantitative  value  as  it  should.  The  model  is 
verified. 
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V. 


Results 


This  chapter  summarizes  the  research,  answers  the  research  and  investigative  questions, 
discuss  the  conclusions,  limitations  and  possible  future  work. 

Summary  of  the  Research  and  Answers  to  Questions 

After  a  detailed  literature  review  thirteen  factors  have  been  found  that  affects  the  selection 
of  PBL  as  an  acquisition  method.  These  factors  are: 

1 .  Reduction  in  total  cost  of  ownership,  life  cycle  cost 

2.  Presence,  Determination  of  metrics,  performance  outcomes 

3.  Improvement  in  performance 

4.  Partnership,  strategic  alliance 

5.  Length  of  contract 

6.  Flexibility 

7.  Better  Service,  Best  Value  Created,  Best  Practices 

8.  Successful  implementation  of  PBL,  PBL  experience  and  training 

9.  Innovation 

10.  Data  availability 

1 1 .  Candidates  and  competition 

12.  Properties  of  system 

13.  Up-front,  early  planning 

This  is  the  answer  for  the  research  question:  ‘What  factors  detennine  the  appropriateness 
of  the  selection  of  PBL  as  a  strategy  for  a  specific  acquisition?’ 

Four  questions  are  asked  as  investigative  questions.  Answers  and  the  processes  for  research 
are  explained  below. 
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What  is  the  current  policy  to  use  PBL?  Regarding  this  policy  does  every  acquisition  fit  in 

PBL? 

DoD’s  policy  is  applying  PBL  whenever  and  wherever  applicable.  It  is  not  only  suggested, 
but  also  mandated  by  DoD  policies  and  acquisition  regulations.  Though,  applicability  is  not 
detailed  much.  This  research  has  a  contribution  from  this  point  of  view.  PBL  does  not  lit  in  every 
acquisition.  There  are  some  cases  where  PBL  is  not  applicable,  where  it  is  not  advised  and  where 
it  is  not  the  best  method. 

Has  DOD  ever  experienced  any  failure/pitfall  due  to  selecting  PBL  as  their  strategy? 

There  are  many  best  practices,  awarded  applications  of  PBL  mentioned  in  the  literature. 
In  addition  to  those  good  examples,  there  are  various  lessons  learned.  But  literature  does  not 
mention  any  failure  regarding  selecting  PBL  as  their  strategy.  PBL  is  mainly  used  in  war  goods 
and  achieved  a  success  somehow;  extremely  or  a  little. 

How  do  the  criteria  effect  acquisition  regarding  PBL? 

As  mentioned  above  there  are  13  factors  detennined  that  affects  the  success  of  PBL.  Each 
factor  has  an  effect  in  its  way.  Till  now,  PBL  has  achieved  an  average  of  25-30%  total  cost 
reduction.  In  addition  a  10%  total  cost  reduction  is  also  categorized  as  a  success.  Therefore  at 
least  a  reduction  should  be  expected,  but  much  is  better. 

Well  detennination  of  metrics  is  one  of  the  most  important  factors.  The  metrics  should  be 
understandable,  clear,  specific,  traceable,  measurable,  and  fair  to  all  partners.  The  better  you 
define  metrics,  the  more  success  you  achieve.  It  is  suggested  to  use  top  level  metrics;  operational 
availability,  operational  reliability,  cost  per  unit  usage,  logistics  footprint  and  logistics  response 
time.  It  is  also  advised  to  use  5  or  less  metrics  not  to  be  complex  in  performance  metrics' 
evaluations. 
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Performance  improvement  is  counted  as  one  of  the  purposes  of  PBL.  Experiences  show 
that  35%  perfonnance  improvement  is  achieved  for  each  metrics  in  average.  But  more  than  10% 
is  also  counted  as  a  success.  Therefore  at  least  an  improvement  should  be  expected,  but  much  is 
better. 

PBL  is  a  partnership,  an  alliance.  PBL  requires  more  sophisticated,  complex  relationships 
than  traditional  methods.  To  build  and  manage  relationships  is  so  important.  At  least  long  tenn 
relationships  including  trust  between  partners,  infonnation,  risk  and  benefit  sharing  is  required. 
The  better  you  manage  and  build  partnering  relationships,  the  more  success  you  achieve. 

For  a  win-win  solution  as  PBL  offers,  a  sufficient  length  of  contract  is  an  obligation. 
Buyer’s  goal  is  to  reach  the  desired  outcomes,  whereas  contractor  tries  to  get  profit  via  return  on 
investment.  Although  there  are  few  examples  of  successful  3  year  PBL  contracts,  at  least  5  years 
contract  is  recommended. 

PBL's  purpose  is  buying  outcomes,  not  dictating  what  to  do.  In  PBL  environment  it  is 
advised  to  tell  the  contractor  what  is  needed,  not  how  to  achieve  it.  As  long  as  the  buyer  gets 
what  s/he  wants,  contractor  has  the  flexibility  how  the  support  is  provided.  Maximum  flexibility 
should  be  provided  for  contractor's  actions.  Buyer  should  let  the  contractor  to  do  what  s/he  does 
best.  PBL  should  allow  partners  to  use  their  knowledge,  innovation,  experience.  As  flexible  as  it 
can  be  will  give  better  results. 

PBL  is  integration  of  partners  to  find  the  way  to  reach  best  practice.  It  is  the  combination 
of  all  organic  and  commercial  abilities.  Managing  life  cycle  systems,  directing  supply  chain 
management,  and  trying  to  meet  the  operational  sustainment  metrics  abilities  accompanying  with 
skills,  knowledge  and  experience  regarding  the  system  are  important.  It  is  also  important  to  find 
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the  best  mix  of  qualified  personnel.  Cooperation  of  organic  and  commercial  elements  should  be 
aligned.  The  more  qualified  service,  the  better  job  is  done. 

PBL  is  a  new  culture,  a  transition  from  traditional  approaches.  Implementation  is  the 
most  important  process.  The  most  effective  way  for  an  implementation  is  making  use  of  others' 
experience.  Awareness,  training,  education  are  requirements  to  be  better.  Subject  matter  experts 
support  a  continuous  education,  training  for  PBL.  The  more  you  leam  the  better  job  you  do. 

Innovation,  creativity  is  a  fundamental  element  of  PBL,  especially  regarding  perfonnance 
metrics  improvement.  Innovation  is  another  skill,  that  contractor  should  have  in  addition  to 
his/her  knowledge  and  experience.  S/he  should  use  the  given  freedom  for  innovative  approach  to 
have  better  results  regarding  metrics.  All  subject  matter  experts  agree  that  innovation  is  a  success 
key  booster  in  PBL.  The  more  innovative  approaches,  the  more  success  you  will  achieve. 

Three  types  of  data  are  required  for  PBL.  This  acquisition  method  is  buying  performance 
according  to  in  advance-set  perfonnance  values.  To  set  those,  historical  data  is  required.  For  cost 
estimations,  cost  data  is  required.  And  when  it  comes  to  payments;  recent  perfonnance  data  is 
required  for  metrics  evaluations.  The  purpose  is  to  have  the  most  accurate,  sufficient  data. 

Competition  is  another  requirement  in  PBL  as  it  is  for  the  other  methods.  Buyer  should 
find  contractors  who  are  willing  to  accept  the  PBL  strategy.  It  is  experienced  that  candidates  are 
not  so  excited  about  PBL.  A  more  competitive  environment  is  desired.  And  in  case  of  having  just 
one  candidate  for  the  acquisition,  OEM  or  prime  vendor  can  achieve  the  performance 
requirements,  although  there  is  not  any  competition.  But  also  it  is  hard  to  manage  many 
suppliers.  Therefore  system  level  PBL  under  an  umbrella  with  a  PSI  is  better  for  success. 
Experiences  show  that  few  suppliers,  such  as  1  to  4,  are  good  for  PBL. 
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PBL  is  mostly  good  for  new,  strategically  important,  high-tech,  complex,  long  life, 
capital  intensive  systems.  And  PBL  is  not  good  at  all  either  for  legacy  systems  at  their  end  of 
lives,  or  for  highly  reliable  systems.  But  PBL  is  recommended  for  capital  intensive  systems.  If 
insourcing  required  lots  of  investments,  PBL  would  rather  be  preferred. 

The  last  factor;  up-front  planning  and  early  PBL  implementation  is  only  applicable  to 
new  systems.  Early  decision  of  PBL,  will  booster  success  of  the  sustainment  program.  Early 
implementation  includes  well  implementation  of  sustainment  processes  regarding  design, 
sustainment  in  the  field,  partnership  and  responsibilities.  The  earlier  you  get  ready  for  PBL,  the 
more  success  you  will  achieve. 

According  to  this  information,  a  tool  is  built  to  assess  the  conformity  of  PBL  as  an 
acquisition  method.  The  tool  assumes  a  linear  relationship  between  factors  and  success  of  PBL. 
Qualitative  measures,  gathered  from  literature  and  user's  answers,  are  converted  to  qualitative 
measures  in  to  have  a  scale  and  a  more  precise  result.  Results  show  that  not  all  acquisitions  are 
fit  in  PBL.  User  can  see  these  results  in  results  spreadsheet  by  numbers,  percentage 
quantitatively  or  by  comments  qualitatively. 

Is  this  assessment  tool  usable  in  every  service-acquisition  decision? 

In  section  4,  applications  of  different  types  of  procurement  are  presented.  All  of  the 
applications  of  the  tool  are  seem  to  be  successful.  Therefore  it  can  be  stated  that  the  tool  is 
applicable  in  almost  every  service-acquisition  decision.  In  addition  to  that  this  tool  might  be 
applicable  to  any  stage  of  procurement  decision.  User  may  apply  it  at  the  beginning  of  decision 
process  to  assess  the  confonnity  of  PBL  as  an  acquisition  method  and/or  it  is  applicable  after  a 
detailed  business  case  analysis  with  more  accurate  values,  answers  and/or  it  is  even  applicable 
after  the  procurement  with  results  to  see  if  PBL  has  achieved  a  real  success. 
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Conclusion 


PBL  is  not  applicable  to  every  acquisition.  But  it  is  possible  to  detennine  the  success  of 
PBL  by  looking  at  the  factors  of  selecting  an  acquisition  method.  This  research  lists  these  factors 
and  explains  how  they  affect  the  success.  The  tool,  built  according  to  this  information,  is  a  handy 
tool  which  will  help  detailed,  time  consuming  business  case  analysis.  It  will  help  in  decision 
making  process  and  save  time  which  is  precious  for  decision  makers.  Not  only  this  research  but 
also  the  tool,  itself  is  a  contribution. 

Limitations 

The  main  limitation  is  the  language,  English;  both  for  the  qualitative  intensive  thesis  and 
creating  questions  to  capture  effects  for  the  tool,  since  it  is  not  the  author's  mother  tongue. 

Interrelationships  between  factors  were  not  taken  into  account,  since  the  regression 
analysis  cannot  be  applicable  to  find  a  regression  model  due  to  lack  of  values  for  dependent 
factor,  success.  Although  binary  values  may  be  applicable  for  success  and  failure,  logistics 
regression  is  not  applicable  either  due  to  lack  of  failed  PBL  examples.  Neither  regression  models 
nor  interactions  terms  were  used,  although  they  may  improve  the  model. 

Although  a  sufficient  literature  review  is  done,  some  questions  in  the  tool  may  not  be 
adequate  to  evaluate  that  factor  in  some  special  cases.  That  is  the  reason  tool  suggests  detailed 
business  case  analysis  even  if  results  are  high  enough.  In  addition  to  that,  some  questions  in  the 
tool  may  beyond  the  knowledge  of  user.  For  these  special  cases  the  tool  may  lack  to  assess  the 
factor.  But  the  tool  is  a  spreadsheet  and  upgradeable  with  more  questions  to  prevent  this 
challenge. 
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Although  a  user  friendly  tool  is  tried  to  be  built,  and  an  instructions  is  implemented,  half- 
an-hour  training,  education  regarding  the  tool,  the  goods/service  in  subject  and/or  PBL  might  be 
required. 

The  tool  evaluation  should  be  repeated  for  every  possible  candidate.  Tool  is  not  capable 
of  evaluating  each  of  them  in  a  spreadsheet. 

Future  Work  Suggestions 

Each  factor  by  itself  is  a  possible  research  topic.  Like  Gardner  (2008)  did  in  contract's 
length,  each  factor  might  be  examined  in  details.  And  according  to  these  studies  the  tool  might 
be  improved  with  many  more  questions  to  fill  the  gaps  explained  in  limitations. 

Another  research  might  be  done  in  selecting  the  best  acquisition  method  by  building  a 
tool  to  assess  every  factor  affecting  all  acquisition  methods  and  compares  them. 
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1.  Reduction  in 

total  cost  of 
ownership,  life 
cycle  cost 

2.  Presence, 

Determination  of 
metrics, 
performance 

outcomes 

3.  Improvement  in 
performance 

4.  Partnership, 
strategic  alliance 

5.  Length  of 

contract 

6.  Flexibility 

7.  Better  Service, 

Best  Value 
Created,  Best 

Practices 

8.  Successful 
implementation  of 
PBL,  PBL  experience 
and  training 

9.  Innovation 

10.  Data  availability 

11.  Candidates  and 
competition 

12.  Properties  of 
system 

13.  Up-front,  early 
planning 

Anonymous  (2005) 

X 

X 

X 

X 

X 

BAE  (2010) 

X 

X 

X 

X 

X 
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X 

X 

X 

X 

X 

X 

Barnes  and  Johnson  (2010) 

X 

X 
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X 

X 

X 

X 
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X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 
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X 

X 

X 

X 

X 

X 

X 

X 

Blumberg  (2006a) 

X 

X 

X 

X 

X 

X 

X 

Blumberg  (2006b) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Blumberg  (2007a) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Blumberg  (2007b) 

X 

X 

X 

X 

X 

X 

X 

Boehk  (2003) 

X 

X 

X 

Bozkurt  and  Guducu  (2005) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Brown  and  Cothran  (2005) 

X 

Burkett  (2008) 

X 

X 

X 

X 

Buyukgural  (2009) 

X 

X 

X 

X 

X 

X 

Canaday  (2006) 

X 

X 

X 

X 

X 

X 

X 

X 

Cebeci  (2009) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Cohen  and  Netessine  (2007) 

X 

X 

X 

X 

X 

Cothran  (2008) 

X 

X 

X 

X 

Cothran  (n.d) 

X 

X 

X 

X 

X 

X 

Dean  (2002) 

Devries  (2004) 

X 

X 

X 

X 

X 

Dibenedetto  (2007) 

X 

X 

X 

X 

X 

X 

DoD  Designing  and  Assessing  Supportabil 

X 

X 

X 

X 

X 

X 

X 

X 

DoDD  5000.01  (2003) 

X 

X 

X 

X 

X 

X 

X 

X 

DoDD  5000.02(2002) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Doerr,  Lewis  and  Eaton  (2005) 

X 

X 

X 

X 

X 

X 

X 

Fogarty  (2006) 

X 

Fowler  (2009) 

X 

X 

X 

X 

X 

X 

Fowler  (2010) 

X 

X 

X 

X 

X 

X 

Gansler,  et  al.  (2011) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

GAO  Report  (2008a) 

X 

GAO  Report  (2008b) 

X 

X 

X 

X 

X 

X 

X 

Gardner  (2008) 

X 

X 

X 

X 

X 

X 

Geary  (2006) 

X 

X 

X 

X 

X 

X 

Geary  and  Vitasek  (2005) 

X 

X 

X 

Geary,  et  al.  (2010) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Gillie  (2006) 

X 

X 

Goure  (2009) 

X 

X 

X 

X 

X 

X 

Goure  (2010) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Goure  (2011a) 

X 
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X 

X 

X 

X 

X 

Harada  (2010) 

X 

X 

X 

X 

X 

X 

X 

Hedden  (2005) 

X 

X 

X 

X 

X 

X 

Hedden  (2007) 

X 

X 

X 

X 

Heller  (2001) 

X 

X 

Heron  (2010) 

X 

X 

X 

X 

X 

Kelman  (2007) 

X 

Kim,  Cohen  and  Netessine  (2006) 

X 

X 

X 

X 

X 

X 

Kobren  (2009) 

X 

X 

X 

X 

X 

X 

X 

Kobren  (2010) 

X 

X 

X 

X 

X 

X 

X 

X 

X 
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1.  Reduction  in 

total  cost  of 

ownership,  life 
cycle  cost 

2.  Presence, 
Determination  of 
metrics, 
performance 

outcomes 

3.  Improvement  in 
performance 

4.  Partnership, 
strategic  alliance 

5.  Length  of 

contract 
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7.  Better  Service, 
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Practices 
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implementation  of 
PBL,  PBL  experience 
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12.  Properties  of 
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Kratz  (2001) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Kratz  and  Buckingham  (2010a) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Kratz  and  Buckingham  (2010b) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Lessons  Learned  (2004) 

X 

X 

X 

X 

X 

Mahadevia,  Engel  and  Fowler  (2006) 

X 

X 

X 

Marietta  (2008) 

X 

X 

X 

X 

Maylettand  Vitasek  (2008) 

X 

X 
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X 

X 

X 

X 

X 

X 
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X 

X 

X 
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X 

X 

X 

X 

Mitchell  (2008) 

X 

X 

Newsome  (2008) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Nicosia  and  Moore  (2006) 

X 

X 

X 

X 

X 

X 

X 

X 

Ott  (2008) 

X 

X 

X 

O  wings  (2010) 

X 

X 

X 

X 

X 

X 

PBL Guide  (2005) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Petti ngill,  et  al.  (2004) 

X 

X 

X 

X 

X 

X 

X 

X 

Slinkard  and  Poleskey  (2006) 

X 

X 

X 

X 

Sols,  Nowick,  and  Verma  (2007) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Spring  (2010) 

X 

X 

X 

X 

X 

X 

Trovato  (2004) 

X 

X 

X 

X 

X 

X 

X 

Venema  (2007) 

X 

X 

X 

Vitasek  (2007) 

X 

X 

X 

X 

X 

Vitasek  and  Geary  (2007) 

X 

X 

X 

X 

X 

X 

X 

Vitasek  and  Geary  (2008) 

X 

X 

X 

X 

X 

X 

X 

X 

Vitasek,  Cothran  and  Rutner^2006a) 

X 

X 

X 

X 

X 

X 

Vitasek,  Cothran  and  Rutner$2007) 

X 

X 

X 

X 

X 

X 

X 

Vitasek,  etal.  (2006b) 

X 

X 

X 

X 

X 

X 

X 

Wuchenich  (2008) 

X 

X 

X 

X 

TOTAL 

60 

56 

52 

49 

41 

39 

38 

34 

28 

27 

24 

17 

16 
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Evaluation  Sheet 


INSTRUCTIONS 


GO  TO  RESULTS 


FACTOR  #  1 :  Reduction  in  total  cost  of  ownership,  life  cycle  cost 


What  is  the  %  of  reduction  expectation  in  total  cost? 


What  i‘i  the  %  of  reduction  e 


What  is  the  %  of  reduction  expectation  in  human  rt 


What  is  the  %  of  reduction  expectation  in  inventory  cost? 


How  much  total  cost  reduction  is  expected? 

SELECT 

How  important  is  'total  cost  reduction'  in  selecting  PBL 
as  an  acquisition  method? 

SELECT 

FACTOR  #2:  Presence,  Determination  of  metrics,  performance  outcomes 


How  accurately  are  the  requirements  of  goods/service  defined? 

SELECT 

How  well  are  die  metrics  aligned  to  organization's  goals? 

SELECT 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

SELECT 

How  realistic,  sound  are  the  metrics? 

SELECT 

How  traceable  are  the  metrics? 

SELECT 

How  measurable  are  the  metrics? 

SELECT 

Are  the  metrics  negotiated  with  candidates? 

SELECT 

How  fair  are  the  metrics  for  all  partners? 

SELECT 

Are  metrics  expressed  in  any  of  those,  below? 

SELECT 

“"Operational  availabilityReadiness 

"“Operational  reliability 

"“Cost  per  unit  usage  Affordability 

"“Logistics  footprint 

"“Logistics  response  timeCycle  time 

Is  a  target  value  set  for  each  metrics? 

SELECT 

How  many  metrics  are  planned  to  use  in  contract? 

SELECT 

How  well  are  the  metrics  determined  regarding 
requirements? 

SELECT 

2 

Overall 

How  important  is  'well  determination  of  metrics'  in 
selecting  PBL  as  an  acquisition  method? 

SELECT 

t 


FACTOR  #3:  Improvement  in  performance 


What  is  the  improvement  expectation  in  %  for  the  1st  metrics? 
What  is  the  improvement  expectation  in  %  for  the  2nd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  3rd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  4th  metrics? 


What  is  the  improvement  expectation  in  %  for  the  5th  metrics? 


What  is  the  average  improvement  expectation  in  %  for  other  metrics? 


FACTOR  #4:  Partnership,  strategic  alliance 


What  type  of  relationship  is  expected  to  be  built  with  provider? 


v-  much  integration  is  expected  to  be  built  widvprovklci-? 


How  trustful  are  the  partners  to  each-ethetf- 


w much- informa tion-is-  planned-to-be  sltared-w-ith-partHCfs-?- 


How  much  of  physical  network  is  planned  to  be  shared  among  partners? 


How.  mue-h  risk  will  be  shated-a 


v  much  ot-benefits-w-ill-be-sltared-a 


How.  well  the.  relationships  a 


laged? 


y  supportive  is  the  manage  me  nt-  to  culture  cha  nge? 


:eptable  is-this-eolture  change- 


FACTOR  #5:  Length  of  contract 


|  What  is  the  contract's  planned  length  in  years? _ 

How  sufficient  is  the  contract's  length  tor  return  on  it 

5  Hew-sufficient  is  the  contmet-s-lengtlt-for-^rovidiHg-e- 


sh  flow  continuity  to  provider?- 


How  sufficient  is  the 


s  length-to-accomplish-a 


n  performance-level  of  the  - 


des-ir-ed-e 


SELECT 


SELECT 


How  much  performance  improvement  is  expected? 

SELECT 

3 

Overall 

How  important  is  'performance  improvement'  in  selecting 
PBL  as  an  acquisition  method? 

SELECT 

f 


f 


How  well  relationships  is  expected  to  be  built,  managed? 

SELECT 

4 

Overall 

How  important  is  'relationship  between  partners'  in 
selecting  PBL  as  an  acquisition  method? 

SELECT 

How  sufficient  is  die  contract’s  length? 

SELECT 

5 

How  important  is  'the  contract's  length'  in  selecting  PBL 
as  an  acquisition  method? 

SELECT 

FACTOR  #6:  Flexibility 


How  flexible  are  the  acquisition  plan  and  the  terms  for  provider’s  actions? 


How  much  freedom  is  provided  for  contrac 

How  much  decentrafeed-is-the  authority? 


How  much  decentralized  are  the  responsibilities? 


How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what'  and 


e  financial,  political,  statutory  barrie 


FACTOR  #7:  Better  Service,  Best  Value  Created,  Best  Practices 


How  well  will  the  expected  service  quality  be  provided  by  organic  structure  in  this 
partnership? 


How  well  will  the  expected  service  quality  bw  provided  by  commercial  structure  in  this 
partnership? 


How  capable  is  the-e 


How  qualified  is  tlie-e 


e  life-cycle  of  systems-? 


e  for  directing  supply  chain?- 


How  knowledgable  is  the  o 


How-1  experienced  is  the  o 


e-about-e 


How  ‘.killed  is  the  o 


H  ow-sk-illed  -is- 1  he-a 


■egarding  job  to  be  dotie? 


rsonnel  regarding  job  to  be  done? 


How  capable  is  the  c 


e  life  cycle  of  s; 


How-qualified-is-the  conmiere-ia.l-stftte.fa 
How  knowledgable  is  the  commercial  si 


e-for-directing  supply-ehain?- 


How-  experieneed-is  -the-e  emmere  ia-l-  si 


How-skilled  in  the  commercial  n 


■r  regarding  job  to  be  done-? 


How-we  11- is-the-w  ork-a  Hoe  atiea-dene-betwet 


seerding-te-best 


SELECT 

SELECT 


SELECT 

SELECT 


|How  flexible  is  the  acquisition  plan  for  provider's  actions? 

SELECT 

How  important  is  'flexibility'  in  selecting  PBL  as  an 
acquisition  method? 

SELECT 

t 


How  well  is  the  expected  service  quality  provided  by  this 
partnership? 

SELECT 

7 

Overall 

How  important  is  'the  expected  service  quality1  in 
selecting  PBL  as  an  acquisition  method? 

SELECT 
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TOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 

How  experienced  is  the  1st  partner  (organic)  in  PBL? 

SELECT 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

SELECT 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

SELECT 

How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

SELECT 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 

SELECT 

How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

SELECT 

How  committed  is  the  managements  for  the  culture  change? 

SELECT 

TOR  #9:  Innovation 

How  constraining  are  the  terms  blocking  innovational  approaches? 

SELECT 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

SELECT 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

SELECT 

1 

People:  How  capable  are  the  employees  regarding  well-known  record  of  generating 
innovative  ideas  in  this  partnership? 

SELECT 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  this  partnership? 

SELECT 

People:  How  important  will  innovation  be  as  a  factor  at  employees'  performance 
evaluations  in  this  partnership? 

SELECT 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this 
partnership? 

SELECT 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this 
partnership? 

SELECT 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers, 
competitors,  suppliers,  etc.? 

SELECT 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for 
new  ideas? 

SELECT 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or 
experiments? 

SELECT 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative 
ideas? 

SELECT 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

SELECT 

How  experienced  is  this  partnership  in  PBL? 

SELECT 

8 

Overall 

How  important  is  'experience  in  PBL'  in  selecting  PBL  as 
an  acquisition  method? 

SELECT 

t 


How  innovative  is  this  partnership? 

SELECT 

9 

Overall 

How  important  is  'innovation'  in  selecting  PBL  as  an 
acquisition  method? 

SELECT 

FACTOR  #10:  Data  availability 


How  adequate  is  historical  data? 

SELECT 

How  accurate  is  historical  data? 

SELECT 

How  adequate  is  cost  data? 

SELECT 

How  accurate  is  cost  data? 

SELECT 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

SELECT 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

SELECT 

How  many  years  of  performance  data  is  in  hand? 

SELECT 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

SELECT 

How  timely  is  the  data  management  as  a  level  of  real  time? 

SELECT 

How  adequate  are  the  data  collection  tools? 

SELECT 

How  sufficient  is  the  market  for  cost  estimation? 

SELECT 

|How  sufficient  are  the  available  data  and  data  collection? 

SELECT 

10 

Overall 

How  important  is  'the  data  availability'  in  selecting  PBL 
as  an  acquisition  method? 

SELECT 

FACTOR  #11:  Candidates  and  competition 


Flow  constraining  are  the  terms  blocking  competition? 


Are  alternative  systems  considered  to  meet  requirements? 


Are  qualified  international  sources  permitted  to  enter? 


How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 


In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 


How  many  partners  are  present  in  PBL  including  buyer? 


t 


How  sufficient  is  the  competition? 

SELECT 

11 

How  important  is  'the  competition'  in  selecting  PBL  as  an 
acquisition  method? 

SELECT 

FACTOR  #12:  Properties  of  system 


Is  contract's  subject  a  new  goods/service? 


How  crtitical  is  the  goods/service  regarding  strategic  importance? 


How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 


How  complex  is  the  goods/service? 


What  is  the  percentage  of  expected  life  left  on  goods/service? 


How  reliable  is  the  goods/service? 


How  much  investment  is  required  for  insourcing? 


How  important/critical  is  the  system? 

SELECT 

12 

Overall 

How  important  is  'the  system's  criticality'  in  selecting 

PBL  as  an  acquisition  method? 

SELECT 

FACTOR  #13:  Up-front,  early  planning 


|  Is  'PBL  decision'  given  early  in  acquisition  of  system? 

SELECT  I 

1 

Is  'sustainment  of  the  system  as  it  is  being  designed'  well-planned  up-front? 

SELECT 

Is  'service  of  the  system  serviced  in  the  field'  well-planned  up-front? 

SELECT 

Is  'partnering  of  industry  and  service  branch  support  personnel  in  sustaining  that  asset' 
well-planned  up-front? 

SELECT 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? 

SELECT 

I 


How  well  is  the  up-front,  early  planning  done? 

SELECT 

13 

Overall 

How  important  is  'up-front,  early  planning'  in  selecting 
PBL  as  an  acquisition  method? 

SELECT 

INSTRUCTIONS 


Ll,li,iiaL‘a 
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Results  Sheet 


GO  TO  EVALUATION 
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1  FACTOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 

8 

Tool  Evaluation 

SELECT  #8 

Occurances' weight  SELECT  #12.1 

Occurances’  weighted  Tool  Score 

#  VALUE! 

User's  Score 

SELECT 

Author's  weight  SELECT  #12.1 

Author's  weighted  Tool  Score 

#  VALUE! 

User’s  weight  #VALUE! 

User's  weighted  Tool  Score 

#  VALUE! 

Occurances’  weighted  User's  Score 

#  VALUE! 

Author's  weighted  User's  Score 

#  VALUE! 

User's  weighted  User's  Score 

#  VALUE! 

t 


FACTOR  #13:  Up-front,  early  planning 
-1  «  Tool  Evaluation  0.0000 

Occurances"  weight 

SELECT  #12.1 

Occurances’  weighted  Tool  Score 

#VALUE! 

User's  Score 

SELECT 

Author’s  weight 

SELECT  #12.1 

Author's  weighted  Tool  Score 

#VALUE! 

User's  weight 

SELECT  #12.1 

User's  weighted  Tool  Score 
Occurances’  weighted  User's  Score 
Author's  weighted  User's  Score 

User's  weighted  User's  Score 

r  #VALUE! 
0.0000 

0.0000 

0.0000 

t 


Instructions  Sheet 


Please  answer  all  the  questions  colored  in  orange  according  to  your  knowledge. 

Answers  will  be  selected  from  drop  down  lists. 

You  do  not  have  to  answerthe  questions  which  are  drawed  striketrough. 

You  can  select  "N/A"  (Not  Applicable)  option  if  you  do  not  have  any  idea  or  the  cases  of  N/A  in  some  questions. 

There  are  Oto  10,  %,  Yes/No  answers.  All  of  them  are  in  drop-down  lists.  0  means  lowest,  10  means  highest  level. 

In  case  of  misselection,  non-selection  the  Result  sheet-general  score  will  give  "#VALUEI",  "#REF!"  failure  or  "SELECT"  "Check  Evaluation  Form"  warnings. 
You  can  check  each  factors'  score  below  general  results  to  see  which  factor  evaluation  is  a  failure. 

Some  questions  should  be  applied  to  all  candidates.  Repeat  evaluation  for  each  candidate  for  better  results. 

Reset  button  in  Evaluation  Form  will  reset  all  cells  to  its  original  values,  ("SELECT"  or  "YES/NO") 

GO  TO  RESULTS  button  in  Evaluation  Form  will  go  to  results  sheet. 

GO  TO  EVALUATION  FORM  button  in  Results  sheet  will  go  to  Evaluation  Form  sheet. 

Top  direction  buttons  in  Evaluation  Form  and  Results  sheet  will  go  top  of  results  sheet. 

Cells  and  sheets  which  are  not  use  for  users  are  locked  and/or  hidden  due  to  security  purposes. 

Password  for  locking/unlocking  is  12345 

Password  is  given  for  modifying  purposes.  Before  modification  be  sure  to  make  a  copy  of  original  document. 


GO  TO  EVALUATION 


GO  TO  RESULTS 


Result  form  will  give  you  an  interpretation  according  to  the  author  weighted  tool  score  and  also  what  should  be  done  for  improving  results. 
Result  form  also  gives  the  other  scores  for  user's  interpretation;  occurances'  or  author  or  user  weighted  tool  and  user  scores. 

User  may  check  each  factor's  score  if  s/he  wishes. 
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Appendix  C:  Evaluation  and  Results 
1.  Mass  transportation  of  Turkish  Armed  Forces  Personnel 


INSTRUCTIONS 


GO  TO  RESULTS 


FACTOR  #  1 :  Reduction  in  total  cost  of  ownership,  life  cycle  cost 


What  is  the  %  of  reduction  expectation  in  total  cost? 

What  is  the  %  of  reduction  expectation  in  conract  cost? 

5% 

What  is  the  %  of  reduction  expectation  in  human  resources  requirement? 

15% 

What  is  the  %  of  reduction  expectation  in  inventory  cost? 

10% 

How  much  total  cost  reduction  is  expected? 

LMore  than  none 

How  important  is  'total  cost  reduction'  in  selecting  PBL 
as  an  acquisition  method? 

IMore  than  none 

FAC 

TOR  #2:  Presence,  Determination  of  metrics,  performance  outcomes 

How  accurately  are  the  requirements  of  goods/service  defined? 

6:Quite  a  bit 

How  well  are  the  metrics  aligned  to  organization's  goals? 

7  More  than  quite  a  bit 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

8:Very 

How  realistic,  sound  are  the  metrics? 

8:Very 

How  traceable  are  the  metrics? 

7:More  than  quite  a  bit 

How  measurable  are  the  metrics? 

6:Quite  a  bit 

2 

Are  the  metrics  negotiated  with  candidates? 

NO 

How  fair  are  the  metrics  for  all  partners? 

3  More  than  a  little 

Are  metrics  expressed  in  any  of  those,  below? 

*Operational  availabilityReadiness 
*Operational  reliability 
*Cost  per  unit  usage  Affordability 
*Logistics  footprint 
*  Logistics  response  timeCycle  time 

YES 

Is  a  target  value  set  for  each  metrics? 

YES 

How  many  metrics  are  planned  to  use  in  contract? 

3 

How  well  are  the  metrics  determined  regarding 
requirements? 

5  Moderate  ly 

2 

Overall 

How  important  is  'well  determination  of  metrics'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 

t 


FACTOR  #3:  Improvement  in  performance 


FACTOR  #4:  Partnership,  strategic  alliance 


What  type  of  relationship  is  expected  to  be  built  with  provider? 


How  much  integration  is  expected  to  be  built  with  provider? 


How  much  information  is  planned  to  be  shared  with  p 


How  trustful  are  the  p 


How-mue-h-of-physieal-n 


ark  is-  planned  to  be-Hraredanrong  partners'; 


How  much  risk  will  be  shared  a 


How  much  of  benefits  will  be  shared  a 


How  well  the  relationships  are  managed? 


How  supportive  is  the  n 


culture  change? 


How  accceptable  is  this  culture  change  a 


g  employe! 


FACTOR  #5:  Length  of  contract 


|  What  is  the  contract's  planned  length  in  years? 


What  is  the  improvement  expectation  in  %  for  the  1st  metrics? 

1% 

What  is  the  improvement  expectation  in  %  for  the  2nd  metrics? 

1% 

What  is  the  improvement  expectation  in  %  for  the  3rd  metrics? 

0% 

What  is  the  improvement  expectation  in  %  for  the  4th  metrics? 

N/A 

What  is  the  improvement  expectation  in  %  for  the  5th  metrics? 

N/A 

What  is  the  average  improvement  expectation  in  %  for  other  metrics? 

N/A 

How  much  performance  improvement  is  expected? 

IMore  than  none 

3 

Overall 

How  important  is  'performance  improvement'  in  selecting 
PBL  as  an  acquisition  method? 

8:Very 

lose  -  Short  Term  Relationship:  2 


How  sufficient  is  the  c 


s  length  for  return  o 


5  |How-suffi&iei»-is--the  contra&t-s-l&ngtlwfet^w&vidiirg-easli-flew-cQntlHuit-y^-pf&vidor-L 


How  sufficient  is  the  contract's  length  to  accomplish  a  certain  performance  level  of  the 

desired  outcomes? 


t 


f 


|How  well  relationships  is  expected  to  be  built,  managed? 

2A  little 

Overall 

How  important  is  'relationship  between  partners'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 

|How  sufficient  is  the  contract’s  length? 

2A  little 

5 

How  important  is  'the  contract's  length’  in  selecting  PBL 
as  an  acquisition  method? 

8:Very 

FACTOR  #6:  Flexibility 


Mg 


How  much  freedom  is  provided  for  contractor's  actions? 


How  much  decentralized  is  the  authority? 


How  much  decentralized  are  the  responsibilities? 


How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what'  and 
'how  to  do's? 


How  constraining  are  financial,  political,  statutory  barriers? 


FACTOR  #7:  Better  Service,  Best  Value  Created,  Best  Practices 


How  well  will  the  expected  service  quality  be  provided  by  organic  structure  in  this 
partnership  ? 


How  well  will  the  expected  service  quality  be  provided  by  commercial  structure  in  this 
partnership? 


lapable  is  the  o 


How  qualified  is  tlie-e 


&  life  eye le  -of  systems? 


e  for  directing  supply  chain?  — 


dedgable  k 


How  experienced  is  the  e 


e-about-s- 


How-skiHed  is  the  o 


Hew-ak-illed-is-the-a 


■egnrding  job  to  be  done? 


g  personnel  regarding  job  to  be  done?  - 


•apable  i‘ 


How  qualified  is  the  commercial  t 


cycle  o 


&  for  directing  supply  ehain?- 


How  know  led  gable  is  the  commercial  si 


How  experienced- is  the  commercial  si 

How  skilled  is  the  commercial  manpo\ 


icture  about  system? 
r-regarding  jobto  be  done?  ■ 


How-well  is-tlie-woi  k-alloeation-done-betwei 


scording-to-best 


1  :More  than  none 


1  More  than  none 


7:More  than  quite  a  bit 


7:More  than  quite  a  bit 


8:Very 


SELECT 

SELECT 


How  flexible  Ls  the  acquisition  plan  for  provider's  actions? 

LMore  than  none 

How  important  is  'flexibility'  in  selecting  PBL  as  an 
acquisition  method? 

7:More  than  quite  a  bit 

How  well  is  the  expected  service  quality  provided  by  this 

7:More  than  quite  a  bit 

7 

Overall 

How  important  is  'the  expected  service  quality'  in 
selecting  PBL  as  an  acquisition  method? 

9More  than  very 
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'TOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 


How  experienced  is  the  1st  partner  (organic)  in  PBL? 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

8  How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 
How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

How  committed  is  the  managements  for  the  culture  change? 


FACTOR  #9:  Innovation 


0:Not  at  all 
1  More  than  none 
0:Not  at  all 
N/A 
2A  little 
2:A  little 
2A  little 


How  constraining  are  the  terms  blocking  innovational  approaches? 

6:Quite  a  bit 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

2A  little 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

2A  little 

People:  How  capable  are  the  employees  regarding  well-known  record  of  generating 
innovative  ideas  in  this  partnership? 

4:Somewhat 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  tliis  partnership? 

0:Not  at  all 

People:  How  important  will  innovation  be  as  a  factor  at  employees'  performance 
evaluations  in  this  partnership? 

3  More  than  a  little 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this 
partnership? 

4:Somewhat 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this 
partnership? 

3  More  than  a  little 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers, 
competitors,  suppliers,  etc.? 

2A  little 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for 
new  ideas? 

IMore  than  none 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or 
experiments? 

0:Not  at  all 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative 
ideas? 

3  More  than  a  little 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

2:A  little 

FAC 

TOR  #10:  Data  availability 

How  adequate  is  historical  data? 

6:Quite  a  bit 

How  accurate  is  historical  data? 

5Moderately 

How  adequate  is  cost  data? 

6:Quite  a  bit 

How  accurate  is  cost  data? 

5Moderately 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

6:Quite  a  bit 

10 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

6:Quite  a  bit 

How  many  years  of  performance  data  is  in  hand? 

2  Years  and  more 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

8:Very 

How  timely  is  the  data  management  as  a  level  of  real  time? 

8:Very 

How  adequate  are  the  data  collection  tools? 

6:Quite  a  bit 

How  sufficient  is  the  market  for  cost  estimation? 

6:Quite  a  bit 

FACTOR  #1 1:  Candidates  and  competition 

How  constraining  are  the  terms  blocking  competition? 

2:A  little 

Are  alternative  systems  considered  to  meet  requirements? 

NO 

Are  qualified  international  sources  permitted  to  enter? 

NO 

11 

How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 

6:Quite  a  bit 

In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 

N/A 

How  many  partners  are  present  in  PBL  including  buyer? 

3 

How  experienced  is  this  partnership  in  PBL? 

IMore  than  none 

8 

Overall 

How  important  is  'experience  in  PBL'  in  selecting  PBL  as 
an  acquisition  method? 

7:More  than  quite  a  bit 

How  innovative  is  this  partnership? 

2A  little 

9 

Overall 

How  important  is  'innovation'  in  selecting  PBL  as  an 
acquisition  method? 

7More  titan  quite  a  bit 

How  sufficient  are  the  available  data  and  data  collection? 

8:Very 

10 

Overall 

How  important  is  'the  data  availability'  in  selecting  PBL 
as  an  acquisition  method? 

8:Very 

How  sufficient  is  the  competition? 

5  Moderately 

11 

Overall 

How  important  is  'the  competition'  in  selecting  PBL  as  an 
acquisition  method? 

7More  titan  quite  a  bit 

FACTOR  #12:  Properties  of  system 

Is  contract's  subject  a  new  goods/service? 

NO 

How  crtitical  is  the  goods/service  regarding  strategic  importance? 

4:Somewhat 

How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 

1  More  than  none 

12 

How  complex  is  the  goods/service? 

1  More  than  none 

What  is  the  percentage  of  expected  life  left  on  goods/service? 

Continious 

How  reliable  is  the  goods /service? 

7  More  than  quite  a  bit 

How  much  investment  is  required  for  insourcing? 

3  More  than  a  little 

L  'partnering  of  indu  tr*  and  ^ c r’  ic c  branch  support  personnel  in  sustaining  that  a~ ^ c t' 

SELECT 

well-planned  up-front? 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? — 

SELECT 

|How  important/critical  is  the  system? 

4:Somewhat 

12 

How  important  is  'the  system's  criticality'  in  selecting 

PBL  as  an  acquisition  method? 

9 More  than  very 

43 


Overa-H- 


How 


I 

the  up-front,  early  planning  done?— 


SELECT 


INSTRUCTIONS 
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GO  TO  EVALUATION 
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1  FACTOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 

q  Tool  Evaluation 

1.1667 

Occurances'  weight 

0.073 

Occurances’  weighted  Tool  Score 

0.0853 

O  User's  Score 

1.0000 

Author's  weight 

0.090 

Author’s  weighted  Tool  Score 

0.1050 

User's  weight 

0.080 

User's  weighted  Tool  Score 

0.0939 

Occurances'  weighted  User's  Score 

0.0731 

Author's  weighted  User's  Score 

0.0900 

User’s  weighted  User's  Score 

0.0805 

t 


2.  Meal  Service  for  Privates 


INSTRUCTIONS 


GO  TO  RESULTS 


FACTOR  #  1 :  Reduction  in  total  cost  of  ownership,  life  cycle  cost 


What  is  the  %  of  reduction  expectation  in  total  cost? 


What  i‘i  the  %  of  reduction  e 


What  is  the  %  of  reduction  expectation  in  human  rt 


What  is  the  %  of  reduction  expectation  in  inventory  cost? 


How  much  total  cost  reduction  is  expected? 

2-A  little 

How  important  is  'total  cost  reduction'  in  selecting  PBL 
as  an  acquisition  method? 

5  Moderate  ly 

FACTOR  #2:  Presence,  Determination  of  metrics,  performance  outcomes 


How  accurately  are  the  requirements  of  goods/service  defined? 

3  More  than  a  little 

How  well  are  die  metrics  aligned  to  organization's  goals? 

6:Quite  a  bit 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

6:Quite  a  bit 

How  realistic,  sound  are  the  metrics? 

4:Somewhat 

How  traceable  are  the  metrics? 

3  More  than  a  little 

How  measurable  are  the  metrics? 

3  More  than  a  little 

Are  the  metrics  negotiated  with  candidates? 

NO 

How  fair  are  the  metrics  for  all  partners? 

4:Somewhat 

Are  metrics  expressed  in  any  of  those,  below? 

YES 

“"Operational  availabilityReadiness 

“"Operational  reliability 

“"Cost  per  unit  usage  Affordability 

“"Logistics  footprint 

“"Logistics  response  time  Cycle  time 

Is  a  target  value  set  for  each  metrics? 

YES 

How  many  metrics  are  planned  to  use  in  contract? 

6 

How  well  are  the  metrics  determined  regarding 
requirements? 

5  Moderate  ly 

2 

Overall 

How  important  is  'well  determination  of  metrics'  in 
selecting  PBL  as  an  acquisition  method? 

10:Extremely 

t 


FACTOR  #3:  Improvement  in  performance 


What  is  the  improvement  expectation  in  %  for  the  1st  metrics? 
What  is  the  improvement  expectation  in  %  for  the  2nd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  3rd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  4th  metrics? 
What  is  the  improvement  expectation  in  %  for  the  5th  metrics? 


What  is  the  average  improvement  expectation  in  %  for  other  metrics? 


FACTOR  #4:  Partnership,  strategic  alliance 


What  type  of  relationship  is  expected  to  be  built  with  provider? 


6 -e-x|ieeted  to  be  -built-with  ■  provide  r-?— 


How  trustful  are  the  partners  to  each  other? 


V-  much- informa tion-is-  planned -to-be  sltared-w-ith-partHCffr?- 


How  much  of  physical  network  is  planned  to  be  shared  among  partners? 


How.  much  risk  will  be  shared-a 


v  much  ot-benefits-w-iH-be-sltared-a 


g-partoers?- 


How  well  the  relationships  . 


Hew-s 


is-the-H 


taged? 


o  culture-  change  ?- 


:eptable  is-this-eolture  change- 


lose  -  Short  Term  Relationship:  2 


FACTOR  #6:  Flexibility 


How  much  decentralized  is  the  authority? 


How  much  freedom  is  provided  for  contractor's  actions? 


How  much  decentralized  are  the  responsibilities? 


How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what1  and 
'how  to  do's? 


How  constraining  are  financial,  political,  statutory  barriers? 


FACTOR  #7:  Better  Service,  Best  Value  Created,  Best  Practices 


How  well  will  the  expected  service  quality  be  provided  by  organic  structure  in  this 
partnership? 


How  well  will  the  expected  service  quality  be  provided  by  commercial  structure  in  this 
partnership? 


How  capable  is  the-e 


How  qualified  is  tlie-e 


e lite-c-yde  of  -systems-?  - 


e  for  directing  supply  chain?- 


How  knowledgable  is  the  o 


How  experienced  is  the  o 


How  skilled  is  the  o 


r  regarding  job  to  be  done? 


How-skilled -is  - the-u 


How  capable  is  the  c 


rsonnel  regarding  job  to  be  done? 

structured*)  manage-hle  cycle- of  .s; 


How  qualified  is  the  commercial  t 


e-for-  directing-  supply-  c  ha  in?  - 


How  knowledgable  is  the  commercial  si 


How  experienced  is  the  commercial  structure  about  system? 


skilled  is  the  commercial  n 


r  regarding  job  to  be  done? 


How  well  is  the  work  allocation  done  betwei 


s  according  to  best  practices? 


How  much  performance  improvement  is  expected? 

IMore  than  none 

3 

How  important  is  'performance  improvement'  in  selecting 
PBL  as  an  acquisition  method? 

6:Quite  a  bit 

1  More  than  none 


1  More  than  none 


1  More  than  none 


9More  than  very 


9More  than  very 


3  More  than  a  little 


8:Very 


SELECT 

SELECT 


SELECT 

SELECT 


t 


How  well  relationships  is  expected  to  be  built,  managed?  | 

2A  little 

Overall 

How  important  is  'relationship  between  partners'  in 
selecting  PBL  as  an  acquisition  method? 

2A  little 

FACTOR  #5:  Length  of  contract 

|what  is  the  contract's  planned  length  in  years? 

1 

How  sufficient  is  the  contract 

:s  length  -for  return  on  in 

vestment  allowance? — 

SELECT 

5  How-sufficient  is  the  contract 

:s-  length-for  providing-cc 

ish-flow-eontinuity-to-provider4 — 

SELECT 

desired-outeemes?- 

~  Lufith  to  accomplish  i 

,L“l‘|Jur" 

SELECT 

f 


How  sufficient  is  die  contract's  length? 

IMore  than  none 

5 

How  important  is  'the  contract's  length'  in  selecting  PBL 
as  an  acquisition  method? 

5  Moderate  ly 

How  flexible  is  the  acquisition  plan  for  provider's  actions? 

IMore  than  none 

How  important  is  'flexibility'  in  selecting  PBL  as  an 
acquisition  method? 

4:Somewhat 

How  well  is  the  expected  service  quality  provided  by  this 
partnership? 

5:Moderately 

7 

Overall 

How  important  is  'the  expected  service  quality'  in 
selecting  PBL  as  an  acquisition  method? 

9More  than  very 
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'TOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 


How  experienced  is  the  1st  partner  (organic)  in  PBL? 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

8  How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 
How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

How  committed  is  the  managements  for  the  culture  change? 


FACTOR  #9:  Innovation 


0:Not  at  all 
0:Not  at  all 
N/A 
N/A 
2A  little 
0:Not  at  all 
1  :More  than  none 


How  constraining  are  the  terms  blocking  innovational  approaches? 

8:Very 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

1  More  than  none 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

0:Not  at  all 

People:  How  capable  are  the  employees  regarding  well-known  record  of  generating 
innovative  ideas  in  this  partnership? 

1  More  than  none 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  tliis  partnership? 

0:Not  at  all 

People:  How  important  will  innovation  be  as  a  factor  at  employees'  performance 
evaluations  in  this  partnership? 

0:Not  at  all 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this 
partnership? 

N/A 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this 
partnership? 

N/A 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers, 
competitors,  suppliers,  etc.? 

0:Not  at  all 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for 
new  ideas? 

N/A 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or 
experiments? 

0:Not  at  all 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative 
ideas? 

0:Not  at  all 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

0:Not  at  all 

FAC 

TOR  #10:  Data  availability 

How  adequate  is  historical  data? 

1  More  than  none 

How  accurate  is  historical  data? 

1  More  than  none 

How  adequate  is  cost  data? 

2:A  little 

How  accurate  is  cost  data? 

2A  little 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

4:Somewhat 

10 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

4:Somewhat 

How  many  years  of  performance  data  is  in  hand? 

Less  than  2,5  Months 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

4:Somewhat 

How  timely  is  the  data  management  as  a  level  of  real  time? 

5  Moderately 

How  adequate  are  the  data  collection  tools? 

4:Somewhat 

How  sufficient  is  the  market  for  cost  estimation? 

6:Quite  a  bit 

FACTOR  #1 1:  Candidates  and  competition 

How  constraining  are  the  terms  blocking  competition? 

1  More  than  none 

Are  alternative  systems  considered  to  meet  requirements? 

YES 

Are  qualified  international  sources  permitted  to  enter? 

NO 

11 

How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 

7  More  than  quite  a  bit 

In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 

OTHER 

How  many  partners  are  present  in  PBL  including  buyer? 

2 

How  experienced  is  this  partnership  in  PBL? 

0:Not  at  all 

8 

Overall 

How  important  is  'experience  in  PBL'  in  selecting  PBL  as 
an  acquisition  method? 

6:Quite  a  bit 

How  innovative  is  this  partnership? 

IMore  than  none 

9 

Overall 

How  important  is  'innovation1  in  selecting  PBL  as  an 
acquisition  method? 

4:Somewhat 

How  sufficient  are  the  available  data  and  data  collection? 

2-A  little 

10 

Overall 

How  important  is  'the  data  availability'  in  selecting  PBL 
as  an  acquisition  method? 

6.‘Quite  a  bit 

How  sufficient  is  die  competition? 

7More  dian  quite  a  bit 

Overall 

How  important  is  'the  competition'  in  selecting  PBL  as  an 
acquisition  method? 

7More  dian  quite  a  bit 

FACTOR  #12:  Properties  of  system 

Is  contract's  subject  a  new  goods/service? 

NO 

How  crtitical  is  the  goods/service  regarding  strategic  importance? 

2:A  little 

How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 

0:Not  at  all 

12 

How  complex  is  the  goods/service? 

0:Not  at  all 

What  is  the  percentage  of  expected  life  left  on  goods/service? 

Continious 

How  reliable  is  the  goods/service? 

8:Very 

How  much  investment  is  required  for  insourcing? 

1  More  than  none 

L  'partnering  of  indu^tr*'  and  nor icc  branch  support  personnel '  u  la*  ihj;  that  a^^ct' 

SELECT 

well-planned  up-front? 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? — 

SELECT 

|How  important/critical  is  the  system? 

2A  little 

12 

How  important  is  'the  system's  criticality'  in  selecting 

PBL  as  an  acquisition  method? 

5  Moderate  ly 

43 


Overa-H- 


How 


I 

the  up-front,  early  planning  done?— 


SELECT 


INSTRUCTIONS 
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3.  ANKA  Turkish  Unmanned  Aerial  Vehicle  Support  Program 


INSTRUCTIONS 


FACTOR  #  1 :  Reduction  in  total  cost  of  ownership,  life  cycle  cost 


What  is  the  %  of  reduction  expectation  in  total  cost?  ■ 

M/A 

What  is  the  %  of  reduction  expectation  in  conract  cost? 

28% 

What  is  the  %  of  reduction  expectation  in  human  resources  requirement? 

5% 

What  is  the  %  of  reduction  expectation  in  inventory  cost? 

5% 

FACTOR  #2:  Presence,  Determination  of  metrics,  performance  outcomes 


FACTOR  #3:  Improvement  in  performance 


What  is  the  improvement  expectation  in  %  for  the  1st  metrics? 
What  is  the  improvement  expectation  in  %  for  the  2nd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  3rd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  4th  metrics? 


What  is  the  improvement  expectation  in  %  for  the  5th  metrics? 


What  is  the  average  improvement  expectation  in  %  for  other  metrics? 


FACTOR  #4:  Partnership,  strategic  alliance 


How  trustful  are  the  partners  to  each  other? 


tship  is 


How  much  integration  is  expected  to  be  built  with  provider? 


How  much  information  is  planned  to  be  shared  with  partners? 


How  much  of  physical  network  is  planned  to  be  shared  among  partners? 


How  much  risk  will  be  shared  among  partners? 


How  much  of  benefits  will  be  shared  among  partners? 


How  well  the  relationships  are  managed? 


How  supportive  is  the  management  to  culture  change? 
How  accceptable  is  this  culture  change  among  employees? 


■Not  Applicable  ■ 


8:Very 


6:Quite  a  bit 


5:Moderately 


5Moderately 


3:More  than  a  little 


7  More  than  quite  a  bit 


9More  than  very 


9More  than  very 
4:Somewhat 


FACTOR  #5:  Length  of  contract 


What  is  the  contract's  planned  length  in  years? 


ct's  length  for  ri 


n  investment  allowai 


How  sufficient  is  the  contract's  length  for  providing  cash  flow  continuity  to  provider? 


s  length  to  accomplish  a 


n  performance  level  of  the — 


desired  outcomes? 


FACTOR  #6:  Flexibility 


FACTOR  #7:  Better  Service,  Best  Value  Created,  Best  Practices 


e  quality  be  provided  by  c 


e  quality  bw  provided  by  c 


rship? 


How  capable  is  the  organic  structure  to  manage  life  cycle  of  systems? 


How  qualified  is  the  organic  structure  for  directing  supply  chain? 


How  knowledgable  is  the  organic  structure  about  system? 


How  experienced  is  the  organic  structure  about  system? 


How  skilled  is  the  organic  manpower  regarding  job  to  be  done? 


How  skilled  is  the  acquisition  personnel  regarding  job  to  be  done? 


How  capable  is  the  commercial  structure  to  manage  life  cycle  of  systems? 


How  qualified  is  the  commercial  structure  for  directing  supply  chain? 


How  knowledgable  is  the  commercial  structure  about  system? 


How  experienced  is  the  commercial  structure  about  system? 


How  skilled  is  the  commercial  manpower  regarding  job  to  be  done? 


How  well  is  the  work  allocation  done  between  partners  according  to  best  practices? 


7  More  than  quite  a  bit 


7  More  than  quite  a  bit 


7  More  than  quite  a  bit 


7  More  than  quite  a  bit 


8:Very 


8:Very 


9More  than  very 


9More  than  very 


8:Very 


9More  than  very 


GO  TO  RESULTS 


How  much  total  cost  reduction  is  expected? 

5  Moderately 

Overall 

How  important  is  'total  cost  reduction'  in  selecting  PBL 
as  an  acquisition  method? 

8:Very 

How  accurately  are  the  requirements  of  goods/service  defined? 

7More  than  quite  a  bit 

How  well  are  die  metrics  aligned  to  organization's  goals? 

8:Very 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

8:Very 

How  realistic,  sound  are  the  metrics? 

7More  than  quite  a  bit 

How  traceable  are  the  metrics? 

7More  than  quite  a  bit 

How  measurable  are  the  metrics? 

8:Very 

Are  the  metrics  negotiated  with  candidates? 

NO 

How  fair  are  the  metrics  for  all  partners? 

6:Quite  a  bit 

Are  metrics  expressed  in  any  of  those,  below? 

YES 

“"Operational  availabilityReadiness 

"■Operational  reliability 

"“Cost  per  unit  usage  Affordability 

"■Logistics  footprint 

■"Logistics  response  timeCycle  time 

Is  a  target  value  set  for  each  metrics? 

YES 

How  many  metrics  are  planned  to  use  in  contract? 

2 

How  well  are  the  metrics  determined  regarding 
requirements? 

7More  than  quite  a  bit 

2 

Overall 

How  important  is  'well  determination  of  metrics'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 

f 


How  much  performance  improvement  is  expected? 

5  Moderately 

3 

How  important  is  'performance  improvement'  in  selecting 
PBL  as  an  acquisition  method? 

7More  titan  quite  a  bit 

1 


|How  well  relationships  is  expected  to  be  built,  managed? 

8:Very 

How  important  is  'relationship  between  partners'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 

f 


How  sufficient  is  the  contract's  length? 

8:Very 

5 

Overall 

How  important  is  'the  contract's  length'  in  selecting  PBL 
as  an  acquisition  method? 

8:Very 

H  n v  flmrihln  nrp  iTir  .  •  ■.  j  i  j  ,i  .  r  T  ^  n 

M/A 

How  much  freedom  is  provided  for  contractor's  actions? 

5Moderately 

How  much  decentralized  is  the  authority? 

5Moderately 

How  much  decentralized  are  the  responsibilities? 

6:Quite  a  bit 

How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what1  and 
'how  to  do's? 

3  More  than  a  little 

How  constraining  are  financial,  political,  statutory  barriers? 

2-A  little 

How  flexible  is  the  acquisition  plan  for  provider's  actions? 

6:Quite  a  bit 

How  important  is  'flexibility'  in  selecting  PBL  as  an 
acquisition  method? 

8:Very 

How  well  is  the  expected  service  quality  provided  by  this 
partnership? 

7More  than  quite  a  bit 

7 

Overall 

How  important  is  'the  expected  service  quality'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 
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'TOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 


How  experienced  is  the  1st  partner  (organic)  in  PBL? 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

8  How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 
How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

How  committed  is  the  managements  for  the  culture  change? 


FACTOR  #9:  Innovation 


2A  little 
2:A  little 
N/A 
N/A 
8:Very 

1  More  than  none 
9More  than  very 


How  constraining  are  the  terms  blocking  innovational  approaches? 

3  More  than  a  little 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

7:More  than  quite  a  bit 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

5:Moderately 

1 

People:  How  capable  are  the  employees  regarding  well-known  record  of  generating 
innovative  ideas  in  this  partnership? 

9More  than  very 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  tliis  partnership? 

7  More  than  quite  a  bit 

People:  How  important  will  innovation  be  as  a  factor  at  employees'  performance 
evaluations  in  this  partnership? 

8:Very 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this 
partnership? 

N/A 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this 
partnership? 

5Moderately 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers, 
competitors,  suppliers,  etc.? 

7  More  than  quite  a  bit 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for 
new  ideas? 

N/A 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or 
experiments? 

8:Very 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative 
ideas? 

7  More  than  quite  a  bit 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

N/A 

FAC 

TOR  #10:  Data  availability 

How  adequate  is  historical  data? 

3  More  than  a  little 

How  accurate  is  historical  data? 

8:Very 

How  adequate  is  cost  data? 

3  More  than  a  little 

How  accurate  is  cost  data? 

6:Quite  a  bit 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

9:More  than  very 

10 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

9More  than  very 

How  many  years  of  performance  data  is  in  hand? 

2  Years  and  more 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

9:More  than  very 

How  timely  is  the  data  management  as  a  level  of  real  time? 

9More  than  very 

How  adequate  are  the  data  collection  tools? 

8:Very 

How  sufficient  is  the  market  for  cost  estimation? 

7More  than  quite  a  bit 

FACTOR  #1 1:  Candidates  and  competition 

How  constraining  are  the  terms  blocking  competition? 

2:A  little 

Are  alternative  systems  considered  to  meet  requirements? 

YES 

Are  qualified  international  sources  permitted  to  enter? 

YES 

11 

How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 

8:Very 

In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 

OEM/PRIME  VENDOR 

How  many  partners  are  present  in  PBL  including  buyer? 

2 

How  experienced  is  this  partnership  in  PBL? 

2'A  little 

8 

Overall 

How  important  is  'experience  in  PBL'  in  selecting  PBL  as 
an  acquisition  method? 

7:More  than  quite  a  bit 

How  innovative  is  this  partnership? 

7More  titan  quite  a  bit 

9 

Overall 

How  important  is  'innovation'  in  selecting  PBL  as  an 
acquisition  method? 

9More  than  very 

How  sufficient  are  the  available  data  and  data  collection? 

5Moderately 

10 

Overall 

How  important  is  'the  data  availability'  in  selecting  PBL 
as  an  acquisition  method? 

9More  than  very 

How  sufficient  is  the  competition? 

9:More  than  very 

11 

Overall 

How  important  is  'the  competition'  in  selecting  PBL  as  an 
acquisition  method? 

5  Moderate  ly 

FACTOR  #12:  Properties  of  system 


Is  contract's  subject  a  new  goods/service? 

YES 

How  crtitical  is  the  goods/service  regarding  strategic  importance? 

8:Very 

How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 

7More  than  quite  a  bit 

How  complex  is  the  goods/service? 

9More  than  very 

What  is  the  percentage  of  expected  life  left  on  goods/service? 

100% 

How  reliable  is  the  goods/service? 

6:Quite  a  bit 

How  much  investment  is  required  for  insourcing? 

9:More  than  very 

FACTOR  #13:  Up-front,  early  planning 


INSTRUCTIONS 


|How  important/critical  is  the  system? 

8:Very 

12 

How  important  is  'the  system's  criticality'  in  selecting 

PBL  as  an  acquisition  method? 

6:Quite  a  bit 

|  Is  'PBL  decision'  given  early  in  acquisition  of  system? 

YES 

Is  'sustainment  of  the  system  as  it  is  being  designed'  well-planned  up-front? 

YES 

Is  'service  of  the  system  serviced  in  the  field'  well-planned  up-front? 

YES 

Is  'partnering  of  industry  and  service  branch  support  personnel  in  sustaining  that  asset' 
well-planned  up-front? 

NO 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? 

YES 

f 


How  well  is  the  up-front,  early  planning  done? 

6:Quite  a  bit 

13 

Overall 

How  important  is  'up-front,  early  planning'  in  selecting 
PBL  as  an  acquisition  method? 

7:More  than  quite  a  bit 

t 
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GO  TO  EVALUATION 
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4.  Turkish  Army’s  Advanced  Armored  Combat  Vehicle  (ACV)  Support  Program 


INSTRUCTIONS 


GO  TO  RESULTS 


FACTOR  #  1 :  Reduction  in  total  cost  of  ownership,  life  cycle  cost 


|  What  is  the  %  of  reduction  expectation  in  total  cost? 


•Whftt-i»-the-°‘n-t>f-i:edne  tion-e 


What  is  the  %  of  reduction  expectation  in  human  rt 


|  What  is  the  %  of  reduction  expectation  in  inventory  cost? 


How  much  total  cost  reduction  is  expected? 

7:More  dtan  quite  a  bit 

Overall 

How  important  is  'total  cost  reduction'  in  selecting  PBL 
as  an  acquisition  method? 

7  More  dtan  quite  a  bit 

FACTOR  #2:  Presence,  Determination  of  metrics,  performance  outcomes 


FACTOR  #3:  Improvement  in  performance 


IWhat  is  the  improvement  expectation  in  %  for  the  1st  metrics? 
What  is  the  improvement  expectation  in  %  for  the  2nd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  3rd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  4th  metrics? 
What  is  the  improvement  expectation  in  %  for  the  5th  metrics? 


What  is  the  average  improvement  expectation  in  %  for  other  metrics? 


FACTOR  #4:  Partnership,  strategic  alliance 


What  type  of  relationship  is  expected  to  be  built  witli  provider? 


How-mueh  ir 


‘xpected  to  be  built  with  provider? 


How  much  information -is  planned -to be-shu red- w ith  p 


How  much  of  physical  network  is  planned  to  be  shared  among  partners? 


wmueh-risk-will  be-oharod-a 


H  ow-mucb-  ot-bene  lila-will-  be-  shared-a- 


11  the  relationships  are  managed? 


o  culture  change? 


How  aceceptable  is  this  culture  change  a 


FACTOR  #5:  Length  of  contract 


What  is  the  contract's  planned  length  in  years? 


sufficient  is  the  c 


cf  s  length  for  ri 


ivcstment  allowai 


How  sufficient  is  the  contract's  length  for  providing  cash  flow  continuity  to  provider? 


How  sufficient  is  the  c 


cf  s  length  to  accomplish  a 


n  performance  level  of  the 


desired  outcomes? 


How  accurately  are  the  requirements  of  goods/service  defined? 

7:More  than  quite  a  bit 

How  well  are  die  metrics  aligned  to  organization's  goals? 

9:More  than  very 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

7:More  than  quite  a  bit 

How  realistic,  sound  are  the  metrics? 

8:Very 

How  traceable  are  the  metrics? 

7:More  than  quite  a  bit 

How  measurable  are  the  metrics? 

9:More  than  very 

Are  the  metrics  negotiated  with  candidates? 

NO 

How  fair  are  the  metrics  for  all  partners? 

7  More  than  quite  a  bit 

Are  metrics  expressed  in  any  of  those,  below? 

YES 

*Operational  availabilityReadiness 

*Operational  reliability 

*Cost  per  unit  usage  Affordability 

*Logistics  footprint 

*Logistics  response  timeCycle  time 

Is  a  target  value  set  for  each  metrics? 

YES 

How  many  metrics  are  planned  to  use  in  contract? 

2 

Partnership:  8 


SELECT 


SELECT 

SELECT 


How  well  are  the  metrics  determined  regarding 
requirements? 

7:More  than  quite  a  bit 

2 

Overall 

How  important  is  'well  determination  of  metrics'  in 
selecting  PBL  as  an  acquisition  method? 

10:Extremely 

f 


1 


f 


How  much  performance  improvement  is  expected? 

4:Somewhat 

3 

How  important  is  'performance  improvement'  in  selecting 
PBL  as  an  acquisition  method? 

4:Somewhat 

How  well  relationships  is  expected  to  be  built,  managed? 

8:Very 

Overall 

How  important  is  'relationship  between  partners'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 

How  sufficient  is  die  contract's  length? 

7 More  dian  quite  a  bit 

5 

How  important  is  ’the  contract's  length'  in  selecting  PBL 
as  an  acquisition  method? 

7More  tiian  quite  a  bit 

FACTOR  #6:  Flexibility 


FACTOR  #7:  Better  Service,  Best  Value  Created,  Best  Practices 


How  well  will  the  expected  service  quality  be  provided  by  organic  structure  in  this 
partnership? 


How  well  will  the  expected  service  quality  be  provided  by  commercial  structure  in  this 
partnership? 


How  capable  is  the  o 


How  knowledgable  is  the  o 


c  structure  to  n 


B  life  cycle  of  systems? 


How  qualified  is  the  o 


e  for  directing  supply  chain? 


c  structure  about  system? 


How  experienced  is  the  o 


c  structure  about  system? 


How  skilled  is  the  o 


c  manpower  regarding  job  to  be  done? 


How  skilled  is  the  a 


rsomiel  regarding  job  to  be  done? 


How  capable  is  the  c 


g  life  cycle  of  s; 


How  qualified  is  the  c 


e  for  directing  s 


How  knowledgable  is  the  commercial  structure  about  system? 


How  experienced  is  the  commercial  structure  about  s; 


How  skilled  is  the  commercial  manpower  regarding  job  to  be  done? 


How  well  is  the  work  allocation  done  betwei 


:ording  to  best  practices? 


i»  ,,,  o p > - iv, I p  nrp  iTip  n r'rnir'itinn  nl'1  n  'ind  the  l ■ 1  frtr  nm-Mdcr*"  netinnr-^ 

}q/A 

How  much  freedom  is  provided  for  contractor's  actions? 

5:Moderately 

How  much  decentralized  is  the  authority? 

6:Quite  a  bit 

How  much  decentralized  are  the  responsibilities? 

6:Quite  a  bit 

How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what1  and 
'how  to  do's? 

6:Quite  a  bit 

How  constraining  are  financial,  political,  statutory  barriers? 

4:Somewhat 

8:Very 


9:More  than  very 


How  flexible  is  the  acquisition  plan  for  provider's  actions? 

6:Quite  a  bit 

How  important  is  'flexibility'  in  selecting  PBL  as  an 
acquisition  method? 

8:Very 

How  well  is  the  expected  service  quality  provided  by  this 
partnership? 

9  More  than  very 

7 

Overall 

How  important  is  'the  expected  service  quality'  in 
selecting  PBL  as  an  acquisition  method? 

5  Moderate  ly 

1 
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'TOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 


How  experienced  is  the  1st  partner  (organic)  in  PBL? 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

8  How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 
How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

How  committed  is  the  managements  for  the  culture  change? 


FACTOR  #9:  Innovation 


0:Not  at  all 
0:Not  at  all 
N/A 
N/A 
2A  little 

1  More  than  none 
6:Quite  a  bit 


How  constraining  are  the  terms  blocking  innovational  approaches? 

4:Somewhat 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

4:Somewhat 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

5:Moderately 

1 

People:  How  capable  are  the  employees  regarding  well-known  record  of  generating 
innovative  ideas  in  this  partnership? 

5Moderately 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  this  partnership? 

6:Quite  a  bit 

People:  How  important  will  innovation  be  as  a  factor  at  employees'  performance 
evaluations  in  this  partnership? 

N/A 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this 
partnership? 

N/A 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this 
partnership? 

N/A 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers, 
competitors,  suppliers,  etc.? 

N/A 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for 
new  ideas? 

7  More  than  quite  a  bit 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or 
experiments? 

9More  than  very 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative 
ideas? 

N/A 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

N/A 

FAC 

TOR  #10:  Data  availability 

How  adequate  is  historical  data? 

4:Somewhat 

How  accurate  is  historical  data? 

4:Somewhat 

How  adequate  is  cost  data? 

6:Quite  a  bit 

How  accurate  is  cost  data? 

6:Quite  a  bit 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

8:Very 

10 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

7  More  than  quite  a  bit 

How  many  years  of  performance  data  is  in  hand? 

1  Year 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

5:Moderately 

How  timely  is  the  data  management  as  a  level  of  real  time? 

5  Moderately 

How  adequate  are  the  data  collection  tools? 

5  Moderately 

How  sufficient  is  the  market  for  cost  estimation? 

5  Moderately 

FACTOR  #1 1:  Candidates  and  competition 

How  constraining  are  the  terms  blocking  competition? 

1  More  than  none 

Are  alternative  systems  considered  to  meet  requirements? 

YES 

Are  qualified  international  sources  permitted  to  enter? 

YES 

11 

How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 

5Moderately 

In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 

OEM/PRIME  VENDOR 

How  many  partners  are  present  in  PBL  including  buyer? 

2 

How  experienced  is  this  partnership  in  PBL? 

IMore  than  none 

8 

Overall 

How  important  is  'experience  in  PBL'  in  selecting  PBL  as 
an  acquisition  method? 

8:Very 

How  sufficient  are  the  available  data  and  data  collection? 

5Moderately 

10 

Overall 

How  important  is  'the  data  availability'  in  selecting  PBL 
as  an  acquisition  method? 

5  Moderate  ly 

How  sufficient  is  die  competition? 

8:Very 

11 

How  important  is  'the  competition'  in  selecting  PBL  as  an 
acquisition  method? 

5:Moderately 

FACTOR  #12:  Properties  of  system 

Is  contract's  subject  a  new  goods/service? 

NO 

How  crtitical  is  the  goods/service  regarding  strategic  importance? 

8:Very 

How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 

6:Quite  a  bit 

12 

How  complex  is  the  goods/service? 

5  Moderately 

What  is  the  percentage  of  expected  life  left  on  goods/service? 

60% 

How  reliable  is  the  goods/service? 

4:Somewhat 

How  much  investment  is  required  for  insourcing? 

2:A  little 

L  'partnering  of  indu^tr*'  and  nor icc  branch  support  personnel '  u  la*  ihj^  that  a^^ct' 

SELECT 

well-planned  up-front? 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? — 

SELECT 

|How  important/critical  is  the  system? 

8:Very 

12 

Overall 

How  important  is  'the  system's  criticality'  in  selecting 

PBL  as  an  acquisition  method? 

8:Very 
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How 


Overall- 


I 

the  up-front,  early  planning  done?— 


SELECT 
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1  FACTOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 

q  Tool  Evaluation 

1.5000 

Occurances'  weight 

0.073 

Occurances’  weighted  Tool  Score 

0.1097 

O  User's  Score 

1.0000 

Author's  weight 

0.090 

Author’s  weighted  Tool  Score 

0.1350 

User's  weight 

0.095 

User's  weighted  Tool  Score 

0.1429 

Occurances'  weighted  User's  Score 

0.0731 

Author’s  weighted  User's  Score 

0.0900 

User’s  weighted  User's  Score 

0.0952 

FACTOR  #9:  Innovation 

Tool  Evaluation 

5.8750 

Occurances’  weight 

0.060 

Occurances'  weighted  Tool  Score 

0.3538 

^  User's  Score 

7.0000 

Author's  weight 

0.060 

Author's  weighted  Tool  Score 

0.3525 

User's  weight 

0.107 

User's  weighted  Tool  Score 

0.6295 

Occurances'  weighted  User's  Score 

0.4215 

Author’s  weighted  User's  Score 

0.4200 

User's  weighted  User’s  Score 

0.7500 

FACTOR  #10:  Data  availability 

-g  Tool  Evaluation  5.4545 

Occurances'  weight 

0.058 

Occurances'  weighted  Tool  Score 

0.3167 

User's  Score  5.0000 

Author's  weight 

0.055 

Author's  weighted  Tool  Score 

0.3000 

User’s  weight 

0.060 

User's  weighted  Tool  Score 

0.3247 

Occurances'  weighted  User’s  Score 

0.2903 

Author's  weighted  User's  Score 

0.2750 

User's  weighted  User’s  Score 

0.2976 

FACTOR  #11:  Candidates  and  competition 

1  1  Tool  Evaluation  10.0000 

Occurances'  weight 

0.052 

Occurances'  weighted  Tool  Score 

0.5161 

A  A  User's  Score  8.0000 

Author's  weight 

0.035 

Author’s  weighted  Tool  Score 

0.3500 

User's  weight 

0.060 

User's  weighted  Tool  Score 

0.5952 

Occurances'  weighted  User's  Score 

0.4129 

Author's  weighted  User's  Score 

0.2800 

User's  weighted  User’s  Score 

0.4762 

FACTOR  #12:  Properties  of  system 

1  ,%  Tool  Evaluation  5.5000 

Occurances'  weight 

0.037 

Occurances’  weighted  Tool  Score 

0.2011 

A  L  User's  Score  8.0000 

Author's  weight 

0.060 

Author's  weighted  Tool  Score 

0.3300 

User's  weight 

0.095 

User's  weighted  Tool  Score 

0.5238 

Occurances'  weighted  User's  Score 

0.2925 

Author's  weighted  User's  Score 

0.4800 

User's  weighted  User's  Score 

0.7619 
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5.  F-35  Joint  Strike  Fighter  (JSF)  Support  Program 


INSTRUCTIONS 


GO  TO  RESULTS 


FACTOR  #  1 :  Reduction  in  total  cost  of  ownership,  life  cycle  cost 


What  is  the  %  of  reduction  expectation  in  total  cost? 


What  in  the-%-of-i:edBetion-e 


What  is  the  %  of  reduction  expectation  in  human  rt 


What  is  the  %  of  reduction  expectation  in  inventory  cost? 


How  much  total  cost  reduction  is  expected? 

6,-Quite  a  bit 

Overall 

How  important  is  'total  cost  reduction'  in  selecting  PBL 
as  an  acquisition  method? 

8:Very 

FACTOR  #2:  Presence,  Determination  of  metrics,  performance  outcomes 


How  accurately  are  the  requirements  of  goods/service  defined? 

8:Very 

How  well  are  die  metrics  aligned  to  organization's  goals? 

8:Very 

How  clear,  understandable,  solid,  specific  are  the  metrics? 

9:More  than  very 

How  realistic,  sound  are  the  metrics? 

8:Very 

How  traceable  are  the  metrics? 

9More  than  very 

How  measurable  are  the  metrics? 

8:Very 

Are  the  metrics  negotiated  with  candidates? 

NO 

How  fair  are  the  metrics  for  all  partners? 

8:Very 

Are  metrics  expressed  in  any  of  those,  below? 

YES 

“"Operational  availabilityReadiness 

“"Operational  reliability 

*Cost  per  unit  usage  Affordability 

“"Logistics  footprint 

“"Logistics  response  time  Cycle  time 

Is  a  target  value  set  for  each  metrics? 

YES 

How  many  metrics  are  planned  to  use  in  contract? 

6 

How  well  are  the  metrics  determined  regarding 
requirements? 

8:Very 

2 

Overall 

How  important  is  'well  determination  of  metrics'  in 
selecting  PBL  as  an  acquisition  method? 

10:Extremely 

t 


FACTOR  #3:  Improvement  in  performance 


What  is  the  improvement  expectation  in  %  for  the  1st  metrics? 
What  is  the  improvement  expectation  in  %  for  the  2nd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  3rd  metrics? 


What  is  the  improvement  expectation  in  %  for  the  4th  metrics? 
What  is  the  improvement  expectation  in  %  for  the  5th  metrics? 


What  is  the  average  improvement  expectation  in  %  for  other  metrics? 


FACTOR  #4:  Partnership,  strategic  alliance 


What  type  of  relationship  is  expected  to  be  built  with  provider? 


v-  muck  integration  is  ■expected  to-be  built  with  ■provider-?' 


How  trustful  are  the  partners  to  each-ethetf- 


y-muclt-information-is-  planned-to-be  sltared-w-ith-partHCfs-?- 


How  much  of  physical  network  is  planned  to  be  shared  among  partners? 


How-much  risk- will  be  -shared-a 


v  much  ot-benefits-w-ill-be-slrared-a 


How-'  well  the  relationships  a 


taged?-- 


^  supportive  is  the- management- to  culture-change? 


Hew-aeeceptable  is-thk-e-okure  change- 


FACTOR  #5:  Length  of  contract 


■oko 


ed-tet 


|How  sufficient  is  the  contract's  length  for  return  on  investment  allowance? 


5  |How  sufficient  is  the  contract's  length  for  providing  cash  flow  continuity  to  provider? 


How  sufficient  is  the  contract's  length  to  accomplish  a  certain  performance  level  of  the 
desired  outcomes? 


FACTOR  #6:  Flexibility 


How  much  freedom  is  provided  for  contractor's  actions? 
How  much  decentralized  is  the  authority? 


How  much  decentralized  are  the  responsibilities? 


How  constraining  are  the  acquisition  plan  and  the  terms  with  specifications,  'what1  and 
'how  to  do's? 


How  constraining  are  financial,  political,  statutory  barriers? 


FACTOR  #7:  Better  Service,  Best  Value  Created,  Best  Practices 


How  well  will  the  expected  service  quality  be  provided  by  organic  structure  in  this 
partnership? 


How  well  will  the  expected  service  quality  be  provided  by  commercial  structure  in  this 
partnership? 


How  capable  is  the-e 


How  qualified  is  tlie-e 


8  hfe'Oy&te  of -systems-? 


e  for  directing  supply  chain?- 


How  knowledgable  is  the  o 


How  experienced  is  the  o 


e-about-s- 


How  skilled  is  the  o 


H  ow-sk-iUed  -is- 1  he-a 


•egarding  job  to  be  done? 


rsonnel  regarding  job  to  be  done? 


How  capable  is  the  c 


e  life  cycle  of  s; 


Hovr-qualified-is-tbe  commercial  t 


e-for-directing  supply-ehain?- 


v  knowledgable  is  the  commercial  si 


How-  experieneed-is  ■the-eommere  ia-l-  si 


How-skilled  in  the  commercial  n 


■r  regarding  job  to  be  done? 


How-well-  is-the-w  ork-a  Hoe  ation-done-betwei 


i-eerding-te-best 


Partnership:  8 


n  very 


10:Extremely 


8:Very 


9:More  than  very 
7:More  than  quite  a  bit 


8:Very 


1  More  than  none 


3  More  than  a  little 


5:Moderately 


9More  than  very 


SELECT 

SELECT 


How  much  performance  improvement  is  expected? 

6,-Quite  a  bit 

3 

Overall 

How  important  is  'performance  improvement'  in  selecting 
PBL  as  an  acquisition  method? 

lOrExtremely 

f 


How  well  relationships  is  expected  to  be  built,  managed? 

7:More  than  quite  a  bit 

Overall 

How  important  is  'relationship  between  partners'  in 
selecting  PBL  as  an  acquisition  method? 

8:Very 

f 


How  sufficient  is  die  contract's  length? 

7More  dian  quite  a  bit 

5 

Overall 

How  important  is  'the  contract's  length'  in  selecting  PBL 
as  an  acquisition  method? 

9More  dian  very 

|How  flexible  is  the  acquisition  plan  for  provider's  actions?  |  7:More  than  quite  a  bit 

Overall 

How  important  is  'flexibility1  in  selecting  PBL  as  an 
acquisition  method? 

6:Quite  a  bit 

How  well  is  the  expected  service  quality  provided  by  this 
partnership? 

7:More  than  quite  a  bit 

7 

Overall 

How  important  is  'the  expected  service  quality1  in 
selecting  PBL  as  an  acquisition  method? 

7More  dian  quite  a  bit 

t 
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'TOR  #8:  Successful  implementation  of  PBL,  PBL  experience  and  training 


How  experienced  is  the  1st  partner  (organic)  in  PBL? 

How  experienced  is  the  2nd  (commercial)  partner  in  PBL? 

How  experienced  is  the  3rd  (commercial)  partner  in  PBL? 

8  How  experienced  are  the  other  commercial  partners  in  PBL  as  an  average? 

How  much  importance  is  given  to  educational  programs/training  for  PBL  in  partnership? 
How  trained/knowledgable  is  the  acquisition  personnel  about  PBL? 

How  committed  is  the  managements  for  the  culture  change? 


2A  little 

9:More  than  very 
N/A 

_ N/A _ 

7:More  than  quite  a  bit 
5:Moderately 
9:More  than  very 


FACTOR  #9:  Innovation 


How  constraining  are  the  terms  blocking  innovational  approaches? 

3  More  than  a  little 

How  incentivizing  are  the  terms  encouraging  innovational  approaches? 

6:Quite  a  bit 

How  much  freedom  is  present  to  encourage  innovational  approaches? 

7:More  than  quite  a  bit 

People:  How  capable  are  the  employees  regarding  well-known  record  of  generating 
innovative  ideas  in  this  partnership? 

8:Very 

People:  How  important  is  screening  innovation  skills  in  hiring  process  in  tliis  partnership? 

N/A 

People:  How  important  will  innovation  be  as  a  factor  at  employees'  performance 
evaluations  in  this  partnership? 

N/A 

Process:  How  frequently  will  brainstorming  and  generating  new  ideas  take  place  in  this 
partnership? 

N/A 

Process:  How  important  will  'challenging  the  status  quo/conventional  ways'  be  in  this 
partnership? 

7  More  than  quite  a  bit 

Process:  How  interactive  will  this  partnership  be  for  seeking  opportunities  with  customers, 
competitors,  suppliers,  etc.? 

N/A 

Process:  How  spreaded  will  'the  networking  through  outside  of  this  partnership'  be  for 
new  ideas? 

N/A 

Process:  How  supportive  will  this  partnership  be  for  adopting  pilot  programs  or 
experiments? 

8:Very 

Philosophies:  How  encouraging  will  this  partnership  be  to  employees  to  offer  creative 
ideas? 

N/A 

Philosophies:  How  supportive  will  this  partnership  be  to  risk  taking  for  new  ideas? 

N/A 

FAC 

TOR  #10:  Data  availability 

How  adequate  is  historical  data? 

6:Quite  a  bit 

How  accurate  is  historical  data? 

5Moderately 

How  adequate  is  cost  data? 

2A  little 

How  accurate  is  cost  data? 

7:More  than  quite  a  bit 

How  adequate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

2A  little 

10 

How  accurate  will  the  expected  recent  data  be  for  metrics'  evaluation? 

7  More  than  quite  a  bit 

How  many  years  of  performance  data  is  in  hand? 

Less  than  2,5  Months 

How  attainable  will  the  expected  recent  data/values  of  metrics  be? 

8:Very 

How  timely  is  the  data  management  as  a  level  of  real  time? 

9More  than  very 

How  adequate  are  the  data  collection  tools? 

8:Very 

How  sufficient  is  the  market  for  cost  estimation? 

1  More  than  none 

FACTOR  #11:  Candidates  and  competition 

How  constraining  are  the  terms  blocking  competition? 

3  More  than  a  little 

Are  alternative  systems  considered  to  meet  requirements? 

YES 

Are  qualified  international  sources  permitted  to  enter? 

YES 

11 

How  sufficient  is  'the  number  of  candidates'  willing  to  enter  PBL  partnership? 

8:Very 

In  case  of  sole  candidate;  is  the  candidate  OEM/Prime  Vendor  or  other? 

N/A 

How  many  partners  are  present  in  PBL  including  buyer? 

2 

How  experienced  is  this  partnership  in  PBL? 

5  Moderate  ly 

8 

Overall 

How  important  is  'experience  in  PBL'  in  selecting  PBL  as 
an  acquisition  method? 

lOExtremely 

How  innovative  is  this  partnership? 

6:Quite  a  bit 

9 

Overall 

How  important  is  'innovation1  in  selecting  PBL  as  an 
acquisition  method? 

8:Very 

How  sufficient  are  the  available  data  and  data  collection? 

6:Quite  a  bit 

10 

Overall 

How  important  is  'the  data  availability'  in  selecting  PBL 
as  an  acquisition  method? 

9More  than  very 

How  sufficient  is  die  competition? 

6,-Quite  a  bit 

11 

How  important  is  'the  competition'  in  selecting  PBL  as  an 
acquisition  method? 

b.'Quite  a  bit 

FACTOR  #12:  Properties  of  system 


Is  contract's  subject  a  new  goods/service? 

YES 

How  crtitical  is  the  goods/service  regarding  strategic  importance? 

10:Extremely 

How  technologically  advanced  are  the  requirements  regarding  the  goods/service? 

10:Extremely 

How  complex  is  the  goods/service? 

9More  than  very 

What  is  the  percentage  of  expected  life  left  on  goods/service? 

100% 

How  reliable  is  the  goods/service? 

5  Moderately 

How  much  investment  is  required  for  insourcing? 

8:Very 

FACTOR  #13:  Up-front,  early  planning 


INSTRUCTIONS 


How  important/critical  is  the  system? 

9:More  than  very 

12 

How  important  is  'the  system's  criticality'  in  selecting 

PBL  as  an  acquisition  method? 

7More  dian  quite  a  bit 

|  Is  'PBL  decision'  given  early  in  acquisition  of  system? 

YhS 

Is  'sustainment  of  the  system  as  it  is  being  designed'  well-planned  up-front? 

YES 

Is  'service  of  the  system  serviced  in  the  field'  well-planned  up-front? 

YES 

Is  'partnering  of  industry  and  service  branch  support  personnel  in  sustaining  that  asset' 
well-planned  up-front? 

NO 

Is  'determination  of  the  multiple  responsibilities'  well-planned  up-front? 

YES 

f 


How  well  is  the  up-front,  early  planning  done? 

5:Moderately 

13 

Overall 

How  important  is  'up-front,  early  planning'  in  selecting 
PBL  as  an  acquisition  method? 

5  Moderately 

t 
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157 


158 


Appendix  D:  Study  Board 


An  Assessment  Tool  of 

Performance  Based  Logistics  Appropriateness 


OBJECTIVE: 

Finding  the  factor*  that  afTect  selection  of 
PBL  as  an  acquisition  method  and  building  a 
spreadsheet  tool  according  to  these  factors. 


RESEARCH  QUESTION: 

What  factors  determine  the  appropriateness 
of  the  selection  of  PBL  as  a  strategy  for  a 
specific  acquisition? 


INTRODUCTION: 

PBL  is  determined  product  support  strategy. 
Using  PBL  is  mandated  by  Dol),  regulated  by 
FAR,  suggested  by  Defense  Acquisition 
University. 

Basic  policy  is  applying  PBL  whenever 
wherever  possible.  But  not  all  acquisitions  for 
service*  can  be  conducted  in  a  performance- 
based  manner,  t  here  are  exceptions. 

In  contrast  with  DoD's  policy  there  is  not 
much  about  these  exceptions  in  the  PBL 
literature. 


CONTRIBUTION: 

Determining  these  exceptions  by  building  the 
assessment  tool  which  includes  the  factors 
and  their  afTects. 

This  tool  ss  ill  help  in  decision  making 
processes.  Business  Case  Analysis. 

That  will  save  decision  makers  time. 

That  will  give  more  objective  results. 


OVERVIEW: 

81  documents  are  taken  as  sample  literature. 

Questions  are  prepared  according  to  this 
literature. 

Tool  applied  to  5  different  acquisitions  in 
Turkey. 


CONCLUSION: 

PBL  is  not  applicable  to  every  acquisition.  Rut  it  is  possible  to  determine  the 
success  of  PBL  by  looking  at  the  factors  of  selecting  PBL  as  an  acquisition 
method.  This  research  lists  these  factors  and  explains  boss  they  afTect  the 
success.  The  tool,  built  according  to  this  information,  is  a  handy  tool  svhich 
will  help  detailed,  time  consuming  business  case  analysis,  in  decision  making 
process  and  save  time  which  is  precious  for  decision  makers.  Not  only  this 
research  but  also  the  tool,  itself  is  a  contribution. 


FUTURE  RESEARCH: 

Each  factor  is  a  possible  research  topic.  They  might  be 
examined  in  details.  And  according  to  these  studies,  the  tool 
might  be  improved  with  many  more  questions. 

A  tool  might  be  created  for  selecting  the  best  acquisition 
method  by  assessing  every  factor  affecting  all  acquisition 
methods  and  comparing  them. 


4-AFIT 
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FACTORS  EFFECTING  SUCCESS  OF  PBL 

1 .  Reduction  in  total  cost  of  ownership,  life  cycle  cost 

2.  determination  or  metrics,  performance  outcomes 

3.  improvement  in  performance 

4.  Partnership,  strategic  alliance 

5.  Length  of  contract 
8.  Flexibility 

7.  Better  service,  best  value  creation,  best  practices 

8.  Successful  implementation  ot  PBL.  PBL  experience 

9.  Innovation 
lOData  availability 

1 1  .Candidates  and  competition 

12. Properties  of  system 

13. Up*front.  early  planning 
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